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I. 


INTRODUCTION 


This  project  compares  the  academic  experience  of  Naval  Postgraduate  School 
(NPS)  students  across  two  different  delivery  modes,  namely  the  MBA  resident  and 
distance-learning  (DL)  programs.  This  project  identifies  differences  in  student  outcomes 
for  DL  as  compared  to  resident  students  in  an  attempt  to  address  a  highly  debated  topic  of 
interest  in  higher  education.  Specifically,  this  study  examines  between-group  differences 
in  four  evaluation  areas,  entry  requirements,  grade  point  averages  (calculated  to  assess 
course  and  program  specific  variation),  graduation  status,  and  student  survey  responses 
on  program  experience,  particularly  focusing  on  students’  academic  achievement  and 
program  perceptions.  Project  findings  inform  more  in-depth  studies  of  effective  DL 
courses  and  programs  for  military  and  defense  personnel,  and  may  contribute  to  the 
development  of  higher-quality  computer-assisted  education. 

Accredited  higher  education  is  an  essential  requirement  that  can  better  equip  those 
serving  in  military  and  working  for  the  department  of  defense  to  prepare,  enhance 
readiness,  and  meet  the  nation’s  current  and  future  defense  needs.  Expanding  access  to 
quality  higher  education  among  military  officers  and  defense  personnel  by  delivering 
courses  and  programs  through  DL  instructional  modes  could  increase  their  opportunities 
for  preparedness  and  mission  readiness.  DL  instruction  could  further  allow  officers  to 
complete  their  degrees  in  a  timely  manner,  without  geographic  limitations,  and  provide  a 
continuing  ability  to  fulfill  their  assignments  at  their  duty  stations. 

Like  civilian  institutions,  the  military  appreciates  the  reduced  financial  costs  made 
possible  by  online  higher  education  instruction  (Fodor  2016).  However,  the  military  also 
shares  the  need  to  assess  and  improve  DL  program  quality,  especially  as  compared  with 
traditional  resident  programs.  In  addressing  these  needs,  two  outcomes  are  especially 
important:  students’  achievement  as  shown  by  factors  recorded  in  academic  transcripts, 
and  student  survey  responses  on  their  academic  program  experience.  This  MBA  project 
addresses  these  two  outcomes  through  one  question:  What  are  the  similarities  and 
differences  in  the  MBA  academic  experience  of  resident  and  DL  NPS  students?  To 
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answer  this  question,  I  conducted  a  review  and  analysis  of  data  of  NPS  MBA  students 
who  began  their  NPS  MBA  programs  in  academic  years  (AY)  2006-2013. 

A  review  of  research  findings  conducted  in  civilian  institutions  of  higher 
education  provides  important  background  information  on  differences  in  course  or 
program  success  observed  for  resident  and  DL  students.  Furthermore,  some  studies 
reviewed  were  specific  only  to  MBA  courses  or  programs.  For  example,  one  relatively 
recent  study  by  Harmon,  Alpert,  and  Lambrinos  (2014)  showed  little  or  no  difference  in 
student  success  levels  by  delivery  model  for  coursework  that  requires  minimal  math.  In 
an  earlier  study  also  conducted  in  civilian  institutions,  Bruce  (2010)  found  DL  students 
rated  the  quality  of  their  learning  experience  as  above  average.  However,  Fodor  (2016), 
whose  study  was  conducted  on  NPS  student  populations  across  all  programs,  reports  that 
NPS  DL  students  have  lower  overall  GPAs.  Similarly,  Bacolod  and  Chaudhary  (2016), 
whose  study  examines  NPS  student  achievement  in  overlapping  degree  programs,  report 
that  two  success  outcomes,  grade  point  average  and  graduation,  are  lower  for  NPS  DL 
students  in  programs  with  a  resident  and  DL  option. 

NPS’s  Graduate  School  of  Business  and  Public  Policy  (GSBPP),  one  of  four 
schools  at  NPS,  is  the  only  school  currently  offering  defense-focused  MBA  degrees. 
GSBPP  offers  these  degrees  through  six  resident  and  three  DL  programs,  for  a  total  of  9 
degrees  that  encompass  18  specialized  programs  designed  to  meet  the  needs  of  military 
and  defense  personnel.  For  this  MBA  project,  I  collected  and  analyzed  data  for  GSBPP 
MBA  resident  and  DL  student  cohorts  who  began  in  AY2006  through  AY2013.  I 
grouped  the  student  data  to  obtain  representative  student  samples  identified  by  program 
instruction  mode,  gender  and  U.S.  military  or  civilian  status. 

Four  student  evaluation  factors  were  compared  after  collecting  and  analyzing  the 
student  data  sample:  NPS  program  entry  requirements,  known  as  Academic  Profile  Codes 
(APC),  three  different  NPS  specific  grade  point  averages  (TQPR,  CQPR,  CoreQPR), 
student  graduation  rates,  and  student  feedback  from  the  Graduating  Student  Survey 
(GSS).  These  factors  were  examined  using  t-tests  comparing  difference  in  means.  These 
MBA  project  analyses  did  not  control  for  other  factors  correlated  with  delivery  mode  and 
outcomes. 
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Findings  from  the  project  analyses  for  the  four  student  evaluation  factors  were 
mixed.  For  example,  t-tests  examining  the  different  grade  point  averages  showed  that 
between-group  differences  for  resident  and  DL  MBA  students  were  mixed,  depending  on 
how  these  averages  were  calculated  (i.e.,  all  courses,  MBA  program  required  courses, 
courses  required  by  both  resident  and  DL  MBA  programs).  However,  for  all  three  grade 
point  average  types,  the  cumulative  mean  grade  point  averages  of  MBA  DL  students 
were  higher  than  those  of  resident  program  students.  These  results  align  with  some  of  the 
findings  in  the  literature  reviewed,  that  DL  students  perform  typically  as  well  as  resident 
students  in  less  math  intensive  programs  (Ni  2013).  But,  graduation  rates  were  lower  for 
the  DL  MBA  program  students  aligning  with  research  completed  by  Fodor  (2016)  and 
Bacolod  and  Chaudhary  (2016),  which  observed  NPS  DL  students  across  all  programs 
are  less  likely  to  graduate  than  NPS  resident  students.  In  the  review  of  selected  responses 
from  the  graduating  student  survey  (GSS),  MBA  students  in  DL  programs  rated  their 
academic  experience  higher  than  their  peers  enrolled  in  resident  programs. 

This  MBA  project  is  comprised  of  six  chapters,  organized  as  follows.  Chapter  I 
presents  an  introduction  and  the  purpose  of  my  project.  Chapter  II  provides  the  history 
and  background  of  the  institution  and  the  MBA  programs.  Chapter  III  reviews  the 
literature  relevant  to  resident  and  DL  MBA  student  success.  Chapter  IV  describes  the 
data  analyzed,  and  explains  the  methodology  used  for  statistical  tests  employed  in 
analyzing  the  data.  Chapter  V  discusses  the  results  of  these  statistical  tests.  Chapter  VI 
presents  the  conclusions,  limitations,  and  recommendations. 
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II.  INSTITUTIONAL  BACKGROUND 


A.  MBA  HISTORY  AND  NPS 

The  Naval  Postgraduate  School  (NPS)  was  established  as  the  School  of  Marine 
Engineering  at  the  U.S.  Naval  Academy  in  1909,  and  renamed  the  Naval  Postgraduate 
School  in  1912  (Naval  Postgraduate  School  [NPS]  2016).  In  the  mid-20th  century,  NPS 
was  legislated  to  be  a  degree  awarding,  accredited  institution.  In  1949,  NPS  was 
reorganized  and  relocated  to  Monterey,  California,  from  the  Naval  Academy  in 
Annapolis,  Maryland.  Two  years  later,  after  moving  500  students  and  100  faculty 
members,  NPS  was  up  and  running  again,  offering  graduate  level  education  to  military 
officers  and  defense  personnel  in  its  new  Monterey  location  (NPS  2016). 

The  Graduate  School  of  Engineering  and  Applied  Sciences  (GSEAS)  was  the 
primary  foundation  of  NPS.  GSEAS  remains  one  of  the  four  official  schools  that 
comprise  NPS  to  this  day.  The  GSEAS  school  mission,  also  unchanged,  focuses  on 
science  and  technology  to  increase  the  technical  capability  of  the  Navy  and  U.S.  military 
forces  (Naval  Postgraduate  School  GSEAS  2016).  With  its  move  to  Monterey,  NPS 
expanded  to  include  the  first  Operations  Research  curriculum,  the  founding  basis  of  the 
Graduate  School  of  Operational  and  Applied  Sciences  (Naval  Postgraduate  School 
GSOIS  [NPS  GSOIS]  2016).  The  GSOIS  focuses  on  operations  and  supports  the  military 
in  a  changing,  information  intensive  environment  (NPS  GSOIS  2016). 

Five  years  later  NPS  instituted  its  first  Management  curriculum,  the  foundation  of 
today’s  Graduate  School  of  Business  and  Public  Policy  (Naval  Postgraduate  School 
GSBPP  [NPS  GSBPP]  2016).  The  GSBPP  provides  an  education  in  public  policy  with  a 
defense  focus  to  enrolled  U.S.  military  officers  and  defense  civilians  (NPS  GSBPP  2016). 
NPS  continued  its  growth  in  1972  by  creating  the  first  National  Security  curriculum:  this 
curriculum  helped  launch  the  School  of  International  Graduate  Studies  (Naval 
Postgraduate  School  SIGS  2016).  The  SIGS  focuses  on  international  relations  and 
regional  security  to  meet  the  nation’s  complex  security  needs  (Naval  Postgraduate  School 
Office  of  Institutional  Research  [NPS  OIR]  2006). 
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The  Education  Study  Committee  in  1973  advised  NPS  “to  determine  the  role  of 
graduate  education  in  preparing  Naval  officers  for  the  future”  (quoted  in  Miller  2012,  8). 
In  response,  NPS  established  the  NPS  Continuing  Education  Office  in  1974  Miller  (2012) 
reports,  shortly  after,  NPS  launched  a  group  distance  learning  (DL)  education 
correspondence  courses.  The  1989  NPS  catalog  states  the  Continuing  Education  office 
was  to  become  a  “means  of  providing  extended  educational  services  that  will  more 
comprehensively  fulfill  the  school’s  assigned  mission”  (quoted  in  Miller  2012,  8).  Due  to 
the  office’s  strong  emphasis  on  preparatory  refresher  courses  in  an  attempt  to  increase 
students’  entry  requirement  APC  scores,  program  completion  rates  were  less  than  5%  and 
sadly  the  office  was  closed  in  1989  (Miller  2012). 

The  department  of  aeronautics,  with  a  new  reinvigorated  perspective,  offered  its 
first  graduate  level  DL  course  in  1990  (Miller  2012).  This  effort  was  somewhat  more 
successful  with  13%  course  completion  rates  according  to  a  memo  written  by  Professor 
Ball  in  1993,  which  was  reported  by  Miller  (2012).  This  course  established  the  evolution 
of  DL  at  NPS.  Owen  (2003)  affirms  shortly  following  the  mild  success  of  this  course,  an 
unidentified  but  enthusiastic  thesis  student  studied  how  video-teleconferencing 
technology  (VTE)  might  support  DL  education  at  NPS.  He  found  that  not  only  did  NPS 
have  professors  with  the  knowledge  required  to  set  up  a  VTE  system,  but  more 
importantly  it  was  cost  effective. 

Two  experts  on  campus,  one  a  department  chair  and  the  other  a  dean,  set  out  to 
change  the  history  of  DL  at  NPS  by  beginning  their  search  for  video-conferencing 
technologies  that  would  recreate  a  resident  classroom  for  students  off-site  (Miller  2012). 
By  the  end  of  the  academic  year  in  1994,  Miller  (2012)  notes  they  had  engineered  and 
built  two  VTE  classrooms  on  campus,  recreating  the  on-site  resident  classroom  for  off¬ 
site  DL  students,  these  professors’  work  set  the  course  for  the  first  DL  Masters  of  Science 
in  Aeronautical  Engineering.  Two  years  later,  a  MS  in  Software  Engineering  via  the 
Department  of  Computer  Science  began  enrolling  students.  Both  programs  were 
officially  published  in  the  1998  NPS  Academic  Catalog  (Miller  2012). 

In  an  effort  to  improve  quality  and  access,  NPS  needed  to  advance  and  expand 

beyond  the  physical  constraints  of  its  Monterey  resident  campus.  In  1999,  through  the 
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GSBPP  schoolhouse,  the  first  of  several  online  degree  programs  were  offered  to  U.S. 
military  and  defense  civilians  (NPS  OIR  2006).  The  GSOIS  and  GSEAS  schoolhouses 
soon  followed  suit,  offering  degree  programs  to  DL  students  in  2000,  in  addition  to  an 
expansion  of  degree  program  offerings  from  GSBPP  in  the  same  year  (Naval 
Postgraduate  School  Office  of  the  Registrar  [NPS  OR]  2016).  This  expansion  continued 
across  the  four  schools  over  the  years,  increasing  the  Navy’s  potential  reach  for 
workforce  development,  and  for  incentivizing  officers  with  additional  educational 
benefits.  Further,  online  degree  programs  enhance  NPS’s  ability  to  provide  graduate 
education  to  U.S.  military  officers  unbound  by  geographic  limitations  to  better  meet  the 
ever  changing  needs  of  the  mission. 

Each  NPS  school’s  curriculum  was  developed  chronologically  to  meet  the 
evolving  mission  needs  of  the  Navy  and  U.S.  military.  Not  until  2001  did  the  four  schools 
become  officially  established  as  the  Graduate  School  of  Engineering  and  Applied 
Sciences,  the  Graduate  School  of  Operational  and  Applies  Sciences,  the  Graduate  School 
of  Business  and  Public  Policy,  and  the  School  of  International  Graduate  Studies  (NPS 
OIR  2006).  The  GSBPP  still  provides  the  only  MBA  program  specifically  designed  to 
offer  a  defense-focused,  graduate-level  education  in  business  (NPS  GSBPP  2016).  The 
first  NPS  Masters  of  Science  in  Management  (MSM)  degree  had  been  awarded  in  1960; 
however,  in  2002  most  resident  MSM  degree  programs  were  changed  to  MBA  programs 
upon  the  establishment  of  the  GSBPP  (Coughlan,  Hager,  and  King  2013). 

Each  NPS  MBA  degree  program  has  three  component  parts,  two  of  which  share  a 
common  set  of  core  courses  that  are  standardized  across  all  the  GSBPP  degree  programs. 
The  two  shared  components  are  the  Defense  Management  Core  (52  credit  hours)  and  a 
capstone  project  or  thesis.  The  third  component  differs  for  each  degree  program,  and  is 
comprised  of  a  specialized  Curricular  Concentration  (24  or  more  credit  hours).  The 
GSBPP  offers  five  distinct  resident  MBA  programs,  each  with  its  own  specialized 
component:  Acquisition  Management  (815/816),  Defense  Management  (809/818/820), 
Financial  Management  (834/835),  Information  Management  (870),  or  Logistics 
Management  (814/819/827). 
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Each  GSBPP  MBA  program  has  an  overarching,  independent  educational 
objective,  with  its  curriculum  designed  to  meet  that  objective  and  fulfill  Navy  mission 
needs.  The  MBA  programs  achieve  their  respective  objectives  by  tailoring  each 
curriculum  to  meet  the  educational  requirements  of  specific  Military  subspecialties,  and 
Military  occupational  and  functional  areas  (NPS  GSBPP  2016).  The  MBA  programs 
focus  their  management  education  on  U.S.  military  relevance,  while  interchangeably 
fulfilling  accreditation  standards.  NPS  GSBPP  (2016)  programs  are  accredited  by  the 
Western  Association  of  Schools  and  Colleges  (WASC)  and  two  additional  premier 
accrediting  bodies,  the  Association  to  Advance  Collegiate  Schools  of  Business  (AACSB) 
and  the  National  Association  of  Schools  of  Public  Affairs  and  Administration 
(NASPAA).  This  accreditation  standard  is  met  only  by  a  handful  of  graduate  business 
schools  in  the  U.S.  and  international  higher  education,  (NPS  GSBPP  2016). 

In  addition  to  its  MBA  programs,  NPS  offers  a  resident  Master  of  Science  in 
Management  program  in  two  specialized  components,  Manpower  Systems  Analysis  (847) 
and  Defense  Systems  Analysis  (817).  In  January  of  2000,  the  school  added  two  new  DL 
degree  programs  to  enroll  on  a  part-time  basis.  One  of  those  programs,  the  Master  of 
Science  in  Program  Management  (MSPM)  degree  program  (835),  was  designed  and 
implemented  to  give  students  tools  to  manage  programs  and  projects  efficiently  and 
effectively.  Also  in  January  2000,  a  second  DL  degree  program,  the  Master  of  Science  in 
Contract  Management  degree  program  (MSCM)  (836),  was  constructed  to  equip  students 
with  the  knowledge  required  to  effectively  manage  complex  acquisitions  and  contracts 
(NPS  GSBPP  2016). 

In  June  2002,  following  in  the  footsteps  of  the  MSPM  DL  degree  programs,  NPS 
created  its  first  executive  MBA  DL  degree  program  for  Naval  officers  (NPS  GSBPP 
2016).  This  program  allowed  naval  students  stationed  in  the  U.S.  or  deployed  at  sea  to 
obtain  an  executive  MBA  without  geographical  restriction.  Seven  years  later,  NPS 
launched  a  second  executive  MBA  DL  degree  program  designed  for  Department  of  the 
Navy  (DON)  civilians.  These  two  programs  were  designed  to  merge  the  experience  of 
Navy  officers  and  DON  civilians  with  general  management  and  MBA  core  coursework 
(NPS  GSBPP  2016).  In  total,  GSBPP  provides  nine  Master  degree  programs 
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encompassing  18  active  curricula  that  share  a  core  group  of  courses  relating  to  the 
practice  of  public  management.  Within  GSBPP,  there  are  a  total  of  seven  resident  MBA 
specialized  degree  programs,  and  two  DL  EMBA  programs. 

B.  NPS  COURSE  DELIVERY  MODES 

From  AY2006-AY2013,  the  Naval  Postgraduate  School  offered  several  course 
delivery  modes.  In  the  face-to-face  format,  only  one  course  delivery  mode  was  offered 
for  resident  degree  programs.  By  contrast,  several  different  course  delivery  modes  were 
made  available  for  DL  program  students.  Some  of  these  delivery  modes  have  been 
retired,  but  those  still  active  include  DL- Asynchronous,  DL-Synchronous,  Hybrid  and 
Offsite.  Each  of  these  is  discussed  further  in  the  following  paragraphs. 

All  Naval  Postgraduate  School  students,  whether  resident  or  DL  students,  have 
access  to  NPS’s  open  source  learning  management  system,  called  Sakai.  This  online 
collaborative  learning  and  research  portal  offers  flexibility  for  students  and  faculty.  Sakai 
can  provide  a  course  syllabus,  homework  assignments,  access  to  assigned  readings, 
discussion  board  capabilities  that  track  and  notify  the  posting  of  comments,  video 
lectures,  and  timed  quizzes  and  tests.  These  and  other  Sakai  features  may  be  used  by 
faculty  to  facilitate  better  understanding  of  course  requirements  and  more  opportunities  to 
meet  course  objectives  for  both  resident  and  DL  students.  This  portal  also  provides  an 
online  means  by  which  instructors  can  provide  resident  students  with  both  required  and 
supplementary  course  materials. 

DL-Asynchronous  courses  are  taught  with  materials,  lectures,  exams  and  other 
resources  available  via  Sakai  and  other  online  platforms  for  lectures.  Students  in 
asynchronous  courses  can  access  Sakai  at  any  time  convenient  for  their  schedule.  Sakai 
thus  provides  one  of  the  platforms  that  DL  students  must  access  to  participate  in  classes. 
This  portal  contains,  at  a  minimum,  access  to  course  materials,  exams,  other  resources, 
and  potentially  lectures.  Other  online  platforms  NPS  uses  for  lectures  include  Elluminate, 
a  web-based  software,  and  video  teleconferencing  (VTE),  both  of  which  are  available 
through  NPS  on-campus  classroom  connections  to  off-site  classrooms. 
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DL-Synchronous  online  classes  require  students  and  instructors  to  be  online  at  the 
same  time  to  participate.  That  is,  the  lectures,  discussions,  and  presentations  occur  at  a 
specific  hour,  just  as  in  a  face-to-face  course  format.  VTE  and  Elluminate  are  used  to 
provide  support  to  this  off-site  resident  classroom  experience.  As  shown  by  the  NPS 
Student  Management  Database  and  distance  learning  reports,  all  MBA  courses  are  video 
streamed  using  VTE  technology  (Naval  Postgraduate  School  Office  of  Institutional 
Research  [NPS  OIR]  2016).  The  hybrid  courses  are  taught  partially  online  and  partially  in 
face-to-face  meetings  on  the  NPS  campus.  Offsite  courses,  however,  are  taught  face-to- 
face  at  locations  other  than  the  NPS  campus.  All  instruction  for  resident  courses,  by 
contrast,  takes  place  face-to-face  within  NPS  classrooms. 

Concurrently  with  the  evolution  of  computer  assisted  programs  at  NPS,  public 
and  private  higher  education  institutions  also  began  with  online  course  delivery  modes, 
including  one  NPS-MIT  partnership  (Miller  2012). As  the  use  of  online  platforms  became 
wide  spread,  efforts  to  assess  effectiveness  and  compare  their  outcomes  with  those  of 
traditional  resident  programs  were  undertaken.  In  response  to  such  accessible  online 
learning  sites  such  as  the  Kahn  academy,  reports  and  entire  journals  were  devoted  to  DL 
education  were  common. 
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III.  LITERATURE  REVIEW:  STUDENT  SUCCESS  IN  NPS 
RESIDENT  AND  DL  MBA  PROGRAMS 


This  chapter  provides  an  overview  of  academic  research  on  the  advantages  and 
drawbacks  reported  for  resident  and  distance  learning  (DL)  programs  in  the  literature.  It 
also  reports  on  literature  findings  for  assessing  DL  programs  and  students,  as  well  as  the 
assessment  of  MBA  students  in  DL  programs,  and  comparisons  of  resident  and  DL 
student  success. 

A.  HISTORY  OF  EARLY  DISTANCE  LEARNING  AND  THE  POTENTIAL 

FOR  EDUCATING  MILITARY  AND  DEFENSE  PERSONNEL 

Although  distance  learning  (DL)  is  commonly  considered  to  be  the  product  of 
20th  century  computer  technologies  applied  to  a  range  of  instructional  objectives, 
scholars  report  that  it  has  been  attempted  with  every  technological  advance  since  the 
invention  of  the  printing  press  (Frick  1991).  According  to  Harting  and  Erthel  (2005),  for 
example,  the  first  instance  of  DL  education  in  the  United  States  was  offered  almost  three 
hundred  years  ago  in  the  form  of  correspondence,  designed  to  help  students  to  master 
new  technologies.  To  attract  students  to  this  DL  opportunity,  the  instructor  used  the  mail 
and,  in  March  1728,  a  Boston  newspaper  to  advertise  courses  in  shorthand  (Holmberg 
1995,  Harting  and  Erthel  2005). 

The  first  correspondence  school.  The  Society  to  Encourage  Studies  at  Home,  was 
established  in  the  1800s  by  a  woman.  Ana  Eliot  Ticknor  (Harting  and  Erthel  2005).  This 
enterprising  teacher  and  entrepreneur  provided  classes  in  over  20  different  subjects.  Her 
students  were  primarily  young  women  who  had  duties  at  home,  as  was  typical  of  the  time 
period  (Harting  and  Erthel  2005).  The  next  offering  of  a  correspondence  program  came 
about  through  the  Chautauqua  movement,  a  byproduct  of  the  Lyceum  movement,  both  of 
which  provided  specific  formats  for  forums  aimed  at  adult  education.  As  a  supplement  to 
the  Chautauqua  forums,  which  commenced  shortly  after  the  Civil  War,  home-study 
courses  were  provided,  as  well  as  a  home-study  certificate  program  spanning  four  years 
(Harting  and  Erthel  2005). 
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In  1870,  Illinois  Wesleyan  University  became  the  first  university  in  the  United 
States  to  begin  a  DL  correspondence,  home-study  program  (Harting  and  Erthel  2005). 
Two  years  later,  the  first  president  of  the  University  of  Chicago  brought  to  this  institution 
the  correspondence  program  he  had  originated  at  the  NY  Chautauqua  (Harting  and  Erthel 
2005).  Shortly  thereafter,  in  1883,  the  first  correspondence  university  was  launched  in 
Ithaca,  New  York.  By  1892,  Penn  State  had  developed  a  correspondence  program  as  well 
(Banas  and  Emory  1998).  All  these  early  DL  programs  for  higher  education  were 
established  on  the  premise  of  offering  “educational  opportunities  to  students  who  were 
unable  to  participate  in  traditional,  full-time  residency  programs”  (Banas  and  Emory 
1998,  366). 

According  to  Kentor  (2015),  in  the  early  1900s  radio  broadcasting  made  courses 
accessible  via  radio  waves.  The  University  of  Wisconsin  benefited  from  this  new 
technology,  offering  a  handful  of  courses  for  credit  (Kentor  2015).  In  the  1930s,  the 
University  of  Iowa  was  the  first  pioneer  of  television  broadcast  for  education;  three  more 
universities  followed  in  its  footsteps,  including  the  University  of  Michigan  (Kentor 
2015).  By  the  end  of  World  War  II,  moreover,  universities  were  beginning  to  offer 
courses  for  credit  that  were  televised.  Miller  (2012)  reports  that  Coastline  Community 
College,  the  first  U.S.  virtual  community  college,  began  its  operation  by  using  “tele 
courses”  in  1976,  well  before  widespread  access  to  the  Internet  was  available  to  civilians. 

The  1980s  brought  about  the  first  management  programs  and  online  accredited 
graduate  degrees  (Miller  2012).  Moreover,  the  early  expansion  of  Internet  technology 
over  the  1990s  evolved  to  provide  both  real  time  in  sync,  synchronous  instruction,  and 
out  of  sync,  asynchronous  instruction  modes  for  courses,  with  a  corresponding  increase  in 
the  number  of  DL  programs  offered  by  universities  nationwide  (Miller  2012).  Further,  by 
the  mid  1990s,  the  first  fully  web-based  accredited  university  was  established,  paving  the 
way  for  accredited  DL  programs  in  the  21st  century. 

The  Internet  and  technology  boom  of  the  early  21st  century  provided  an 

opportunity  for  all  universities  to  diversify  and  grow  their  DL  degree  programs.  In  the 

late  1990s,  content  management  systems  emerged  and  the  digital  universe  expanded. 

Online  teaching  and  learning  began  to  be  as  easy  to  access  as  the  World  Wide  Web 
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(WWW).  New  approaches  to  DL  platform  applications  were  rapidly  developed,  from 
Blackboard  LMS  in  1997,  to  the  Kahn  Academy  in  2006,  followed  by  YouTube  EDU 
free  lectures  in  2009  and  MOOC  platforms  like  edX,  used  by  Ivy  League  and  other  world 
renowned  technology  schools  (Kaplan  and  Haenlein  2016). 

Technology  that  permits  classes  in  real  time  or  at  reliable  intervals,  offers 
particularly  important  opportunities  for  military  and  defense  personnel.  As  Banas  and 
Emory  (1998)  observed,  “For  the  military  experiencing  budget  constraints... having  to  do 
more  with  fewer  resources... the  old  model  of  sending  people  to  centralized  education 
facilities  for  extended  periods  of  time  makes  rapid  deployment  difficult”  (Banas  and 
Emory  1998,  370-371).  As  organizations  were  created  to  promote  the  use  of  DL  within 
the  Federal  government  in  the  late  1990s,  the  military  joined  in  and  began  its  own 
consideration  of  DL  education  (Banas  and  Emory  1998).  By  making  use  of  the 
communications  systems  vital  to  their  other  operations,  the  military  was  well  positioned 
to  offer  cost-effective  educational  incentives  to  military  and  defense  personnel  (Banas 
and  Emory  1998). 

Taken  together  these  studies  suggest  that,  many  educators  have  contributed  to  the 
development  of  DL  across  the  nation’s  history,  however,  despite  such  enthusiasm  for  the 
potential  benefits  that  DL  programs  provide,  their  quality  has  always  been  questioned, 
allowing  resident  education  to  remain  the  standard.  The  broad  access  provided  by  online 
DL  programs  has  especially  prompted  concerns  about  DL  student  outcomes  as  compared 
with  those  of  students  receiving  a  resident  education.  To  address  these  questions, 
researchers  have  suggested  the  need  to  identify  factors  that  may  differently  affect  resident 
and  DL  students. 

Meanwhile,  outcome  measures  for  student  success  and  program  quality  continue 
to  be  based  on  factors  traditionally  reported  in  the  transcripts  of  resident  students  enrolled 
in  accredited  programs.  That  is,  measures  developed  resident  students  are  applied  and 
used  to  assess  differences  between  resident  and  DL  student  achievement  in  most  reports. 
This  MBA  project  research  makes  use  of  these  same  measurements,  but  also  includes 
program- specific  comments  from  resident  as  well  DL  NPS  MBA  students. 
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B.  IMPACT  OF  COURSE  INSTRUCTION  MODES  ON  STUDENT  SUCCESS 

IN  RESIDENT  AND  DL  MBA  PROGRAMS 

A  number  of  reports  examine  the  different  modes  available  for  course  instruction 
in  resident  and  distance  learning  (DL)  MBA  programs.  These  DL  modes,  which  include 
asynchronous,  synchronous  and  hybrid  approaches,  may  each  introduce  factors  playing 
an  important  role  in  student  and  program  success.  Terry  (2007)  describes  resident  courses 
as  those  courses  instructed  in  person,  face-to-face,  and  at  a  specific  time  and  place  on  a 
traditional  campus.  For  a  3-unit  class,  a  total  of  45  hours  of  face-to-face  instruction 
typically  takes  place.  By  contrast,  DL  asynchronous  (self-paced)  and  synchronous 
(paced)  modes  of  instruction  are  described  as  approaches  in  which  course  instruction 
does  not  occur  on  a  campus  any  longer,  but  instead  converts  the  campus  classroom  into  a 
virtual  one  (Terry  2007).  Terry  (2007)  further  emphasizes  the  potential  to  support  student 
and  faculty  communication  offered  by  a  hybrid  approach,  which  in  his  opinion  combines 
the  best  attributes  of  both  resident  and  DL  courses.  According  to  Terry  (2007,  21),  the 
hybrid  approach  is  characterized  by  “approximately  18  to  25  hours  of  student  and  faculty 
interaction  associated  with  a  3  unit  course”  occurring  in  face-to-face  sessions.  For  the 
remainder  of  the  course,  course  communications  for  the  remainder  of  the  course  are 
substituted  by  online  interaction  and  a  virtual  platform  to  access  course  materials  (Terry 
2007). 

Many  studies  that  examine  overall  outcomes  for  DL  online  programs,  hybrid 
programs,  and  resident  programs  report  course  and  program  findings  differentiated  by 
instruction  modes.  Importantly,  studies  assessing  student  outcomes  for  DL  MBA 
programs  have  reported  that  different  modes  of  DL  course  delivery  may  present  very 
specific  challenges  (Stephens  2007).  Some  of  these  challenges  include  the  inability  to 
verify  academic  integrity,  decreased  student  engagement,  and  students  feeling  more 
isolated  due  to  fewer  opportunities  for  networking  and  connecting  with  peers  (Ni  2013). 

According  to  Ni  (2013),  feelings  of  isolation  and  reduced  connections  can  arise 
with  DL  modes  of  instruction,  particularly  asynchronous  modes.  This  finding  is  of  note 
as  experiences  in  collaboration  and  group  practices  that  best  support  it  are  extremely 
important  for  military  students  (Gratton  and  Erickson  2007).  Programs  designed  for  these 
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students  should  therefore  consider  that  the  development  of  networks  and  connections  for 
comradery  and  team  building,  core  values  of  the  military,  may  prove  more  difficult  when 
students  are  stationed  in  different  places  and  their  only  connection  is  online.  To  assist  in 
better  program  design,  studies  of  military  and  defense  personnel  student  outcomes  that 
are  achieved  through  various  DL  modes  of  instruction  should  be  conducted  to  help 
develop  approaches  better  supporting  core  team  values. 

The  challenges  of  the  virtual  environment  can  also  be  important  for  business 
students,  especially  since  real  world  business  professional  environments  they  hope  to  join 
are  built  around  communications  that  support  collaboration  to  meet  organizational  goals 
(Gratton  and  Erickson  2007).  Group  projects,  for  example,  form  a  major  component  of  all 
MBA  programs,  but  participating  in  such  groups  in  an  online  environment  poses  an 
increased  challenge.  On  the  other  hand,  these  challenges  likely  reflect  interactions  that 
will  occur  after  graduation  in  current,  online  business  environments,  which  place  an 
increasing  emphasis  on  collaboration  (Gratton  and  Erickson  2007). 

Kearns  (2012)  states  that  synchronous  DL  environments  can  both  provide 
numerous  benefits  and  eliminate  some  of  the  problems  reported  for  asynchronous  DL 
environments,  especially  those  barriers  arising  from  being  out  of  sync.  A  synchronous 
class  eliminates  time  delays  in  communications  between  students  and  professors,  as  well 
as  bringing  the  classroom  into  the  home  or  off-site  classroom.  These  advantages,  which 
increase  opportunities  for  shared  experiences  among  DL  students,  can  help  students 
develop  connections  among  peers  and  with  faculty.  According  to  Ni  (2013),  such 
connections  may  reduce  student  feelings  mentioned  of  isolation  and  frustration  within  the 
asynchronous  environment. 

Quintanar  (2012)  believes  that  studies  comparing  resident  and  DL  education  often 
find  the  quantity  of  communicative  interactions  in  DL  to  be  far  less  important  than  their 
quality.  Quintanar  (2012)  also  comments  that  whether  by  synchronous  or  asynchronous 
means,  “the  learning  process  is  developed  through  the  interaction  and  exchange  of  ideas 
in  computer-assisted  communication”  (147).  A  number  of  researchers,  sharing  this  view, 
proceeded  to  examine  a  range  of  factors  contributing  to  quality  DL  instruction,  including 

instructor  opinions  on  and  contributions  to  the  quality  of  DL  education  (Quintanar  2012)  . 
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According  to  Bentley,  Shegunshi,  and  Scanned  (2010)  and  Harmon,  Alpert,  and 
Lambrinos  (2014),  hybrid  modes  of  instruction  provide  the  benefits  of  both  resident  and 
DL  modes.  These  modes  allow  the  communication  and  connections  of  resident  programs, 
but  greatly  reduce  the  face-to-face  time  required,  allowing  full  time  professionals  to 
better  manage  the  time  needed  for  their  courses.  They  also  provide  the  benefits  of  online 
access  for  discussions  after  a  lecture,  homework  submission  and  exams,  without 
eliminating  opportunities  for  connections  (Ni  2013).  For  the  sample  of  graduate  students 
studied  by  Harmon,  Alpert,  and  Lambrinos  (2014),  hybridization  of  courses  and 
programs  counterbalance  the  shortcomings  of  resident  and  DL  instruction. 

As  Internet-based  DL  became  an  accepted  mode  of  instruction,  discussion  of 
student  performance  and  appropriate  performance  measures  for  DL  program  success 
began  to  be  debated  by  program  faculty,  administrators,  researchers  and  accreditation 
organizations.  Comments  in  early  21st  century  reports  began  to  suggest  that  many  factors 
might  specifically  affect  student  outcomes  in  courses  delivered  solely  or  largely  by 
computer- assisted  technologies  (Terry  2007).  Decreased  personal  interaction,  part-time 
student  status,  the  demands  on  working  students,  and  other  obstacles  have  been 
mentioned  as  potentially  more  influential  for  DL  than  for  resident  students  (Terry  2007). 
Nevertheless,  most  assessment  measures  of  DL  student  and  program  success  were 
transferred  directly  from  evaluation  approaches  developed  for  measuring  resident  student 
success  in  traditional  programs. 

The  technological  advances  of  the  1990s  motivated  educators  to  develop  the 
synchronous  and  asynchronous  communication  approaches  for  DL  education  made 
possible  through  computers  (Kumari  2001).  Synchronous  communication  can  be  used  to 
provide  instruction,  such  as  instructors  giving  lectures  “in  sync”  i.e.,  in  real  time,  with 
students  in  different  areas  using  their  computers  to  attend  class  together  despite  in  some 
case  being  in  different  locations  (Kearns  2012).  Asynchronous  communication  provides 
instruction  “out  of  sync,”  with  the  instructor,  for  example,  providing  materials  or  pre¬ 
recording  a  lecture  that  students  can  access  at  their  convenience  (Kearns  2012).  Together, 
these  communication  approaches  provided  the  foundation  on  which  DL  programs  were 
based. 
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C.  PERFORMANCE  MEASURES  USED  TO  EXAMINE  STUDENT 

ACHIEVEMENT  IN  RESIDENT  AND  DL  MBA  PROGRAMS 

The  computer  environment  for  DL  education,  and  the  students  and  faculty  willing 
to  test  its  potential,  changed  more  rapidly  than  assessment  studies  could  keep  up  with 
(Wilkes,  Simon,  and  Brooks  2006).  For  example,  evaluations  of  distance  learning  (DL) 
course  content  relied  primarily  on  1990s  assessment  concepts,  developed  before  DL 
education  was  well  known  (Miller  2012).  The  following  summary  thus  provides  a 
chronological  account  of  research  on  DL  versus  resident  education  assessment,  and  the 
performance  measures  primarily  used  for  these  assessments. 

Program  assessments  that  relied  on  traditional  measures  include,  among  many 
others,  those  by  Navarro  and  Shoemaker  (1999),  Anstine  and  Skidmore  (2005),  and 
Carrol  and  Burke  (2010).  These  researchers  all  evaluate  exam  scores.  Anstine  and 
Skidmore  (2005)  averaged  student  test  scores  for  identical  courses  with  identical 
instructors,  and  report  that  online  students  in  the  MBA  statistics  courses  they  evaluated 
had  lower  exam  scores  than  resident  students.  Although,  they  found  that  DL  students 
perform  worse  than  resident  students  for  statistics,  they  found  no  difference  between 
these  groups  for  the  managerial  courses  (Anstine  and  Skidmore  2005).  One  explanation 
for  the  different  outcomes  observed  between  statistics  and  management  courses  could  be 
differential  DL  and  resident  student  exposure  to  professional  managerial  operations  and 
responsibilities,  a  factor  not  addressed  in  the  report. 

Anstine  and  Skidmore  (2005)  results  aligned  with  current  beliefs  about  DL 
courses  at  the  time.  However,  online  platforms  for  DL  were  rapidly  improving  in  user 
functionality,  overall  efficiency,  and  program  design  across  the  period  of  their  study.  For 
example,  only  two  years  after  the  Anstine  and  Skidmore  (2005)  report,  another  study 
suggested  that  hybrid  programs  yielded  higher  student  grades,  higher  course  evaluation 
scores  and  higher  student  retention  than  did  strictly  online  distance  programs  (Terry 
2007).  This  shift  in  point  of  view  expressed  by  these  researchers  anticipates  the  far 
reaching  impact  of  technological  advances  on  DL  education. 

The  year  following  the  Anstine  and  Skidmore  (2005)  report,  an  assessment  of  the 
perceptions  of  DL  programs  was  published  by  Wilkes,  Simon,  and  Brooks  (2006).  Using 
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survey  research,  these  investigators  evaluated  faculty  and  student  perceptions  of  online 
courses  and  programs  (Wilkes,  Simon,  and  Brooks  2006).  Their  findings  show  that  a 
larger  percentage  of  faculty  surveyed,  as  compared  to  students,  held  a  much  more 
negative  view  of  online  courses.  This  finding  raised  concern  that  “negative  faculty 
attitudes  towards  online  programs  could  hinder  efforts  to  successfully  deliver  quality 
programs”  (Wilkes,  Simon,  and  Brooks  2006,  137). 

However,  in  his  comparison  of  resident  and  DL  students’  outcomes,  Terry  (2007) 
suggests  that  a  reduction  in  the  quality  of  instruction  in  DL  education  could  be  a  direct 
by-product  of  decreased  personal  interaction.  Terry  (2007)  evaluated  course  grades, 
faculty  evaluations,  course  evaluations,  and  project  grades.  For  all  these  measures,  Terry 
(2007)  finds  both  resident  and  hybrid  students  received  higher  marks  than  students  whose 
programs  were  entirely  conducted  by  computer-assisted  technologies.  Terry’s  (2007) 
findings  support  the  view  that  opportunities  for  face-to-face  course  communication  are 
important  for  student  achievement  and  its  evaluation.  Of  note,  these  opportunities  may  be 
particularly  important  for  students  in  hierarchical  organizations  that  depend  on  effective 
communications,  such  as  the  military. 

Gibson  (2008),  who  also  used  exam  scores  to  measure  learning  outcomes  at  the 
course  level  specifically  for  MBA  students,  similarly  finds  that  resident  students  scored 
more  than  3  percentage  points  higher  on  exams  than  online  students  in  the  same  course. 
In  the  following  year,  Larson  and  Sung  (2009)  likewise  used  course  and  exam  grades  as 
their  outcome  measures,  but  found  student  success  in  their  cohorts  to  be  similar  for  both 
online  and  resident  students.  These  researchers  evaluated  the  grades  of  non-MBA 
students  enrolled  in  a  Management  Information  Systems  course,  finding  no  statistically 
significant  difference  between  the  course  grades  of  the  resident,  DL,  and  hybrid  modes  of 
instruction  (Larson  and  Sung  2009).  It  is  not  clear  if  the  different  findings  reported  in 
these  studies  may  have  resulted  from  differences  in  available  technology,  greater  DL 
program  experience,  differences  in  student  experience  with  computer  interfaces,  or  the 
measures  and  cohorts  evaluated. 

A  study  by  Bruce  (2010)  evaluated  responses  to  a  graduating  MBA  student  survey 

that  addressed  students’  satisfaction  with  their  program  as  a  measure  of  success.  Bruce 
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(2010)  observes  that  online  students  rated  the  quality  of  their  learning  experience  as 
higher  than  average.  In  contrast,  resident  students  reported  higher  satisfaction  than  their 
online  peers  in  such  areas  as  curriculum,  program  management,  and  faculty.  That  same 
year,  Bentley,  Shegunshi,  and  Scannell  (2010)  also  used  survey  questions  to  study 
students’  satisfaction  with  a  hybridized  MBA  program.  This  research  group  states  that 
students  in  hybrid  DL  programs  report  higher  satisfaction  with  their  programs  and 
attribute  their  positive  experience  to  hybridization. 

Christensen  and  Nance  (2012)  compare  undergraduate  and  graduate  grade  point 
averages,  in  addition  to  prerequisite  courses,  as  measures  of  preparedness  for  MBA 
coursework  and  as  influences  on  student  success.  They  report  that  undergraduate  and 
graduate  GPAs  are  correlated  and  suggest  that  these  GPAs  are  a  predictor  of  student 
success  in  an  MBA  program.  They  also  report  that  prerequisite  courses  did  not  correlate 
with  student  success,  and  should  not  be  used  as  a  predictor. 

Ni  (2013)  examined  course  grades  and  assessed  learning  objectives.  In  contrast 
with  studies  suggesting  analogous  student  success  for  resident  and  DL  students  (Navarro 
and  Shoemaker  1999,  Larson  and  Sung  2009,  and  Carrol  and  Burke  2010),  Ni  (2013) 
found  that  research  and  math  based  courses  are  slightly  more  challenging  in  an  online 
environment.  Ni  (2013)  found  that  DL  students  receive  lower  grades  for  such  courses 
than  their  resident  counterparts.  This  finding  supports  Anstine  and  Skidmore’s  (2005) 
observations  of  statistics  courses.  However,  Ni  (2013)  also  states  that  online  more  than 
resident  students,  report  finding  their  instructors  effective  in  teaching  critical  thinking  and 
writing  course  objectives.  Ni  (2013)  suggests  that  these  attitudes  appear  to  lead  to  higher 
success  rates  for  online  students. 

Ni  (2013)  supports  the  argument  that  resident  student  outcomes  are  generally 
higher  than  those  of  online  students  at  the  same  course  level.  Her  assessment  of  student 
failure  rates  found  a  6%  higher  failure  rate  for  DL  versus  resident  students.  Ni  (2013) 
then  correlated  course  failure  rates  to  student  attrition  rates,  and  found  that  DL  students 
have  higher  attrition  rates  than  their  resident  counterparts. 
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However,  Harmon,  Alpert,  and  Lambrinos  (2014),  in  an  evaluation  of  course  level 
exam  grades  for  a  MBA  Principles  of  Economics  class,  confirm  results  reported  earlier 
by  Navarro  and  Shoemaker  (1999),  Larson  and  Sung  (2009),  and  Carrol  and  Burke 
(2010).  These  researchers  all  find  student  success  to  be  equal,  regardless  of  the  mode  of 
instruction.  In  contrast,  some  studies  of  MBA  programs  find  that  DL  student  success  is 
equal  to  that  of  resident  students.  Harmon,  Alpert,  and  Lambrinos  (2014)  also  point  out 
that  Carrol  and  Burke  (2010)  and  Navarro  and  Shoemaker  (1999)  studies  do  not  correct 
for  course  self-selection  bias.  Harmon,  Alpert,  and  Lambrinos  (2014)  state  that  they 
observed  no  noticeable  differences  in  student  success  in  their  sample,  and  that  the  DL 
mode  of  instruction  did  not  affect  student  outcomes. 

In  addition,  Harmon,  Alpert,  and  Lambrinos  (2014)  report  that  online  MBA 
students  found  it  more  valuable  to  watch  and  listen  to  lectures  repeatedly,  as  their 
schedule  allowed,  than  to  attend  a  single  lecture  in  a  resident  course.  The  students 
appreciated  DL  opportunities  to  review  material  and  pause  lectures  as  needed.  Their 
views  were  supported  by  another  report  that  year  on  resident  students,  who  saw  being 
unable  to  utilize  a  variety  of  learning  formats  as  a  disadvantage  (Harmon,  Alpert,  and 
Lambrinos  2014).  However,  Harmon,  Alpert,  and  Lambrinos  (2014),  who  focused  on 
student  success  at  the  course  level,  also  report  that  resident  students  scored  more  than  ten 
percentage  points  higher  on  exams  than  online  students  enrolled  in  the  same  MBA 
statistics  course,  supporting  the  findings  of  Anstine  and  Skidmore  (2005)  and  Ni  (2013) 
for  math-intensive  courses. 

Ni’s  findings  on  a  higher  attrition  rate  for  DL  students  were  recently  supported  by 
Lodor  (2016)  in  a  study  evaluating  NPS  students  across  all  programs.  According  to  Lodor 
(2016),  different  results  are  observed  when  comparing  the  success  of  resident  and  DL 
students  across  all  NPS  programs,  as  compared  with  only  GSBPP  programs.  When 
considering  students  across  all  NPS  programs,  DL  students  do  not  perform  as  well  as 
their  resident  counterparts,  and  are  more  likely  to  leave  the  program  without  graduating. 
However,  when  analyzing  only  GSBPP  students,  Lodor  (2016)  finds  that  DL  students 
perform  equally  as  well  as  resident  students. 
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Bacolod  and  Chaudhary  (2016)  examine  students  enrolled  across  all  NPS 
programs.  They  find  a  significant  negative  DL  impact  on  graduation  status,  with  DL 
students  15%  less  likely  to  graduate  than  their  resident  peers  (Bacolod  and  Chaudhary 
2016).  They  also  report  a  significant  0.38  lower  grade  point  average  for  DL  versus 
resident  students  across  all  NPS  programs  (Bacolod  and  Chaudhary  2016). 

In  many  respects,  the  findings  of  all  these  researchers  suggest  the  continued 
importance  of  observations  in  a  2000  report  on  the  community  of  inquiry  model  by 
Garrison,  Anderson,  and  Archer  (2000).  These  early  DL  education  researchers  claim, 
“effective  online  teaching  in  an  MBA  program  is  simply  a  function  of  three  types  of 
presence:  social,  cognitive,  and  teaching  (Arbaugh  and  Hwang  2005,  Al).”  Arguing  that 
online  learning  can  be  just  as  effective  as  resident  learning,  these  researchers  emphasize 
the  instructor’s  role  in  developing  a  collaborative  classroom  presence  in  a  computer 
assisted  course.  These  researchers  define  social  presence  in  the  online  environment  as 
“the  ability  of  learners  to  represent  themselves  socially  and  emotionally  as  ‘real  people’” 
(Garrison,  Anderson,  and  Archer  2000,  89).  Cognitive  presence  is  developed,  these 
researchers  Garrison,  Anderson,  and  Archer  (2000)  suggest,  by  equipping  learners  to 
“sustain  reflection  and  discourse  with  peers”  (89).  thus  allowing  them  to  build  and 
confirm  new  insights.  Teaching  presence  is  defined  by  Garrison,  Anderson,  and  Archer 
(2000)  as  the  “design,  facilitation,  and  direction  of  students’  cognitive  and  social 
processes,”  so  as  to  enhance  the  classes’  ability  to  realize  meaningful  and  educational 
worthwhile  learning  outcomes  (89).  With  these  three  “presences,”  a  learning  environment 
equitable  to  that  of  resident  environments  can  be  created  by  DL  instructors. 

One  important  advance  since  this  early  report,  has  been  the  advent  of  hybrid  DL 
delivery  programs.  This  approach  takes  steps  toward  advancing  the  “presences”  for 
successful  learning  described  by  Garrison,  Anderson,  and  Archer  (2000),  and  may  help  in 
more  insightfully  evaluating  their  impact  in  the  DL  environment. 

D.  ARTICLES  ON  HYBRID  DELIVERY  OF  EDUCATION 

A  new  concept  for  MBA  programs  provides  students  with  a  hybrid  of  resident  and 
DL  instruction.  These  programs  address  many  challenges  that  arise  from  concerns  posed 
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by  accrediting  bodies.  For  example,  the  revised  standards  of  the  Association  to  Advance 
Collegiate  Schools  of  Business  International  (AACSB)  require  “that  programs  ensure 
students’  efforts  in  a  program  are  ‘equivalent  in  terms  of  depth  and  rigor’  across  all 
delivery  modes”  (Association  to  Advance  Collegiate  Schools  of  Business  International 
[AACSB]  2015,  34).  Offering  a  brief  resident  experience  within  the  MBA  program 
provides  benefits  to  students  that  allow  the  program  to  address  important  AACSB  the 
concerns.  According  to  Christ,  Arsenault,  and  Gault  (2015,  32),  accrediting  bodies 
challenge  DL  MBA  programs  on  “assessment,  institutional  engagement  and  adequately 
addressing  important  business  issues  such  as  innovation.”  As  these  researchers  point  out, 
such  challenges  have  resulted  in  many  programs  requiring  students  to  attend  brief 
resident  portions  of  their  programs. 

As  stated  earlier  in  this  review  of  the  literature,  Terry  (2007)  strongly  argues  that 
hybrid  programs  provide  the  most  positive  qualities  of  resident  and  DL  modes  of 
instruction,  offering  students  the  best  of  both  worlds.  DL  students  who  have  an 
opportunity  to  interact  face-to-face  with  faculty  and  peers  can  better  cultivate  a  sense  of 
community.  With  this  learning  community  established,  the  use  of  DL  modules  in  place  of 
class  time  offers  a  better  balance  between  studies,  work,  and  personal  commitments  for 
busy  professionals. 

Christ,  Arsenault,  and  Gault  (2015)  report  on  studies  conducted  of  “all  133 
AACSB  accredited  universities  that  provide  a  DL  MBA  program.  Of  those  133 
universities,  30  required  a  resident  experience,  making  them  partial  hybrid  programs” 
(33).  According  to  Christ,  Arsenault,  and  Gault  (2015),  hybridization  provides 
opportunities  for  the  direct  assessment  of  a  program,  with  a  small  resident  portion  useful 
in  addressing  many  areas  of  concern  examined  by  accrediting  bodies.  For  example,  the 
hybrid  programs  can  address  one  AACSB  concern  by  structuring  part  of  their  resident 
programming  around  a  current,  significant  business  issue.  An  added  benefit  of  the 
resident  portion  of  a  hybrid  program  is  that  it  increases  institutional  engagement  (Christ, 
Arsenault,  and  Gault  2015). 

These  resident  sessions  can  take  place  over  a  weekend,  or  even  through  a  three  to 

five  day  short  course,  permitting  students  who  have  full  time  jobs  to  attend  a  resident 
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session  without  disrupting  their  careers.  Recent  research  further  suggests  that  the  resident 
portion  of  a  hybrid  program  creates  a  face-to-face  connection  between  the  students  that 
can  help  eliminate  feelings  of  isolation,  potentially  reducing  the  student  stress  reported  by 
Ni  (2013)  that  can  occur  in  DL  programs. 

E.  BENEFITS  AND  COSTS  OF  DL 

Both  tangible  and  intangible  costs  are  associated  with  distance  learning  (DL).  Not 
surprisingly,  student  assessment  is  a  highly  debatable  tangible  cost.  According  to  Kearns 
(2012),  DL  MBA  programs  have  difficulty  in  evaluating  student  outcomes  beyond 
grades.  Asynchronous  modes  of  instruction  can  further  increase  the  level  of  difficulty  in 
evaluating  student  outcomes.  As  students  are  not  in  sync  with  the  professor,  delayed 
feedback  can  occur.  Similarly,  in  the  case  of  any  technological  problems,  such  as  a 
computer  breaking  down,  students  may  not  be  able  to  access  their  professors’  feedback  at 
all  (Kearns  2012). 

Nonetheless,  the  intangible  benefits  of  DL  MBA  programs  cited  in  the  literature 
include  successfully  eliminating  barriers,  including  but  not  limited  to  time  and  scheduling 
constraints,  as  well  as  public  speaking  anxiety  (Ni  2013).  Ni  (2013)  also  quotes 
Hackbarth  (1997)  and  others  in  her  research,  commenting  on  the  convenience  of  the  DL 
mode  of  instruction  for  busy  working  professionals  with  full  time  positions.  One  benefit 
DL  instruction  can  provide  is  an  accommodating  schedule,  flexible  in  nature,  potentially 
permitting  a  better  school,  work  and  life  balance  for  these  students.  Another  benefit  of 
DL  instruction  modes  is  the  lessening  of  in-person  communication  barriers.  DL  modes  of 
instruction  can  make  discourse  more  feasible,  allowing  a  timid  student  to  share  an 
opinion.  Ni  (2013)  reasons  that  online  courage  can  make  it  possible  for  a  student  who 
might  never  speak  up  in  a  classroom  to  gain  confidence  in  public  speaking  through  a  DL 
learning  environment.  Ni  (2013)  also  points  out  the  research  of  Warschauer  (2005),  who 
found  online  communications  reduce  pressure  for  timid  students  by  providing  the  time 
they  need  to  process  and  articulate  opinions,  increasing  students’  desire  to  participate  in 
discussions  and  hence  their  participation  in  the  virtual  classroom. 
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In  an  earlier  list  of  DL  mode  of  instruction  benefits,  Terry  (2007)  mentions  the 
aesthetic  value  of  a  consistent  interface,  along  with  a  program’s  ability  to  reach  students 
around  the  world,  reductions  in  time  spent  commuting  to  and  from  school,  and  sometimes 
the  overall  reduced  cost  of  receiving  a  DL  education  as  opposed  to  a  resident  education. 

Many  researchers  note  the  many  benefits  of  DL  instruction  for  those  who  are 
business  professionals  and  those  who  have  difficulty  with  face-to-face  communication 
and  interaction.  However,  findings  by  Harmon,  Alpert,  and  Lambrinos  (2014)  agreed 
with  Kearns’  (2012)  view  that  drawbacks  also  exist.  In  both  of  these  studies,  samples  of 
graduate  students  reported  that  the  largest  drawback  to  being  in  an  asynchronous  DL 
course  was  not  being  able  to  receive  an  immediate  response  to  a  question  from  an 
instructor  during  lecture  (Harmon,  Alpert,  and  Lambrinos  2014;  Kearns  2012).  These 
researchers’  findings  support  the  view  that  barriers  to  a  real  time  exchange  of  information 
can  make  a  concept  more  difficult  to  understand. 

Ni  (2013)  suggests  other  negative  outcomes  of  a  delayed  communication  between 
faculty  and  students  in  DL  instruction.  These  may  include  increased  feelings  of  isolation, 
confusion  and  frustration,  unverifiable  academic  integrity  and  decreased  student 
engagement  (Ni  2013).  Ni’s  (2013)  concerns  were  not  new.  Ware  (2005)  expressed  some 
of  the  same  views,  and  Terry  (2007)  references  a  1998  report  by  McCormack  and  Jones 
(1998),  which  comments  on  the  costs  of  constrained  communication  in  the  online 
classroom. 

According  to  Terry,  research  since  the  late  20th  century  has  suggested  that  the  cost 
of  DL  instruction  can  include  such  factors  as  hardware  and  software  constraints, 
instructors’  opposition  to  moving  towards  new  and  different  instruction  models  and 
approaches,  and  difficulties  in  gauging  a  student’s  critical  thinking  skills  (Terry  2007). 

Many  of  the  details  associated  with  these  factors  have  changed  with  advancing 
technology  and  increasing  student  and  faculty  competence  in  the  online  environment. 
However,  until  more  accurate  measures  of  program,  course  and  student  outcomes  have 
been  developed  for  the  DL  educational  environment,  innovations  able  to  support  its 
benefits  and  minimize  its  cost  will  be  difficult  to  create,  introduce  and  evaluate. 
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F. 


RELEVANT  ARTICLES 


The  following  articles  and  their  research  contributed  important  approaches, 
findings,  and  interpretations  of  data  to  this  project’s  evaluation  of  student  success  factors. 
Terry  (2007)  researches  resident  and  DL  MBA  student  success,  including  the  success  of 
hybrid  students.  Terry  (2007)  uses  multiple  measures  and  finds  that  DL  student  grades, 
course  evaluations  and  other  areas  are  lower  than  those  of  resident  and  hybrid  students. 
However,  in  some  areas,  such  as  student  assignments  focusing  on  business  applications 
and  teambuilding,  Terry  suggests  that  there  is  no  difference  in  these  three  instructional 
modes  and  that  they  are  equitable  (2007).  Terry’s  approach  uses  multiple  measures,  each 
adjusted  against  differences  in  means  by  use  of  the  Kruskal-Wallis  test,  and  so  provides  a 
model  for  increasing  the  validity  of  data  results  and  conclusions. 

Ni  (2013)  evaluates  course  grades  and  survey  responses  to  determine  learning 
effectiveness.  Ni  also  finds  when  evaluating  learning  outcomes,  the  resident  students  feel 
less  positive  about  the  effectiveness  of  their  learning  outcomes  than  their  DL  peers.  Ni  is 
also  careful  to  consider  how  further  analysis  of  the  data  might  help  explain  differences  in 
resident  and  DL  outcomes.  This  is  seen  in  her  examination  of  higher  failure  rates  for  DL 
students,  which  she  determines  is  related  to  these  students’  higher  attrition  rates  (Ni 
2013).  She  also  suggests  factors  needing  further  analysis  that  are  particular  to  the  online 
environment,  such  as  the  opportunity  it  provides  timid  students  to  participate  in 
discussions.  Ni’s  (2013)  report  provides  an  important  example  of  both  evaluating  data 
and  interpreting  it  carefully  in  light  of  other  findings. 

Fodor  (2016)  focuses  on  resident  and  DL  NPS  students  across  all  programs  for 
the  same  start  academic  years  of  this  MBA  project,  2006  through  2013.  Part  of  his  thesis 
evaluates  student  academic  success  data  by  using  propensity  score  matching.  This 
measure  is  different  from  other  literature  measures  proposed,  with  the  exception  of  a 
study  conducted  by  two  NPS  professors  (Bacolod  and  Chaudhary  2016).  Fodor’s  (2016) 
examination  of  academic  success  factors  focuses  on  entry  requirements  (APC),  overall 
NPS  GPA  (TQPR),  and  student  graduation  status.  Fodor  (2016)  finds  when  observing 
NPS  students  across  all  programs  that  most  DL  students  do  not  perform  as  well  as  their 

resident  counterparts  by  any  of  the  academic  success  factors  evaluated.  The  exception 
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noted  is  for  GSPPB  DL  students.  Fodor’s  findings  may  differ  from  those  of  this  MBA 
project,  as  the  population  evaluated  here  consists  only  of  NPS  MBA  students,  but  we 
each  evaluate  in  some  form,  entry  requirements  (APC),  grade  point  averages  (QPR),  and 
graduation  status. 

Bacolod  and  Chaudhary  (2016),  the  NPS  professors  report  similar  results  to  those 
of  Fodor  (2016).  They  too  evaluated  all  NPS  students  during  the  same  academic  year 
time  frame,  and  used  propensity  score  matching  methods  to  assess  student  success  in 
resident  and  DL  programs  (Bacolod  and  Chaudhary  2016).  Their  study  also  focused  more 
deeply  on  differences  seen  by  examining  instructional  mode  and  demographics.  One  part 
of  Bacolod  and  Chaudhary’s  (2016)  work  examines,  for  NPS  DL  students  across  all 
programs,  the  academic  success  factors  of  grade  point  average,  graduation  status,  and 
number  of  thesis  extensions.  From  this  examination,  these  researchers  suggest  that  DL 
students  across  all  NPS  programs  perform  at  a  lower  level  than  their  resident  peers  and 
are  more  likely  to  request  a  thesis  extension  (Bacolod  and  Chaudhary  2016). 

All  these  articles  present  important  data  that  can  contribute  to  better  assessments 
of  DL  MBA  courses  and  programs.  The  articles  that  evaluate  more  than  one  student 
success  measure  seem  to  offer  the  least  bias  and  most  validity,  and  influenced  my 
decision  to  evaluate  more  than  one  outcome.  Articles  that  evaluated  more  than  one  mode 
of  instruction  were  most  relevant  to  post-secondary  education  and  the  future  of 
instructional  delivery.  In  addition,  I  found  the  articles  supporting  hybrid  modes  of 
instruction  to  be  of  great  theoretical  interest.  This  body  of  research  suggests  that  these 
programs  offer  the  best  of  both  resident  and  DL  instruction  can  potentially  eliminate 
issues  currently  hindering  student  success  in  both  DL  and  resident  modes  standing  alone. 

While  findings  in  much  of  the  literature  on  DL  programs  are  mixed,  they 
generally  show  that  resident  MBA  students  perform  as  well  or  better  than  DL  students  at 
both  the  course  and  program  levels.  In  particular  students  in  the  DL  programs  appear  to 
be  at  a  disadvantage  in  their  math  intensive  courses,  which  weighs  down  their  academic 
performance. 
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The  metrics  of  student  success  most  commonly  used  in  the  studies  examined  for 
this  review  of  the  literature  include  exam  grades  and  course  grades.  The  analyses 
performed  for  this  MBA  project  research  will  not  include  course-level  examination 
scores  as  this  data  is  unavailable,  but  they  will  examine  an  additional,  unique  measure: 
student  responses  to  the  NPS  Graduating  Student  Survey  questions  pertaining  to  student 
satisfaction,  engagement,  and  program  preparation. 
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IV.  STUDENT  ACADEMIC  SUCCESS  AND  EXPERIENCE 

FACTORS 

These  analyses  are  preliminary  investigations  intended  to  identify  areas  deserving 
further  examination  by  NPS  MBA  program  faculty  and  administrators.  Toward  this  goal, 
I  collected  data  from  the  NPS  Student  Management  Database  for  resident  and  distance 
learning  (DL)  students  who  began  their  MBA  programs  in  AY2006  through  AY2013.  At 
the  time  the  study  began,  these  years  offered  the  most  recent  complete  data  available. 

Some  of  the  data  analyzed  was  specifically  designed  by  the  NPS  Admissions 
Department  in  2008  to  better  evaluate  military  and  defense  student  preparedness  for  the 
unique  emphases  of  the  NPS  programs  (Susan  Dooley,  personal  communication,  2017). 
In  particular,  the  NPS  APC  entry  requirements  were  designed  to  account  for 
undergraduate  grade  point  average  (GPA),  mathematics,  and  any  engineering  or  technical 
background  the  student  might  possess. 

Data  analyzed  also  included  grade  point  averages  reflecting  the  students’ 
progress;  these  data  were  grouped  by  grade  point  average  for  all  NPS  courses  taken 
(TQPR),  program  specific  courses  (CQPR),  and  core  courses  required  by  all  NPS  MBA 
programs  (CoreQPR).  In  addition  to  these  data,  student  graduation  rates  (grouped  by  on- 
time,  graduated  and  no  graduation  status)  as  well  as  student  responses  to  the  Graduating 
Student  Survey  (GSS)  were  collected  for  analysis.  Descriptions  of  these  academic  entry 
requirements,  success  and  experience  factors  are  defined  further  in  this  chapter. 

A.  NPS  RESIDENT  MBA  AND  DL  EMBA  ACADEMIC  PROFILE  CODE 

(APC):  A  THREE-PART  EXAMINATION 

The  NPS  Admissions  Office  initially  evaluates  a  prospective  candidate’s  official 
undergraduate  transcripts  to  verify  their  legitimacy  as  issued  by  an  accredited  university. 
After  verification  Admissions  further  evaluates  a  candidate’s  eligibility  for  a  program  by 
determining  their  Academic  Profile  Code  (APC).  This  three-digit  code,  according  to  the 
NPS  OR  (2016),  must  be  at  least  a  “345”  for  the  MBA  resident  degree  programs,  except 
for  Financial  Management,  which  bears  the  greatest  similarity  to  the  DL  EMBA 
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programs,  and  also  requires  a  “245.”  These  programs  also  have  the  greatest  overlap  in 
course  requirements. 

1.  APC  Digit  1:  Undergraduate  GPA 

According  to  NPS  Admissions  Director,  Susan  Dooley  (personal  communication, 
2017),  all  undergraduate  GPAs  are  recalculated  by  Admissions  Office  staff  to  include 
failed  and  retaken  courses.  She  also  reported  that  the  recalculated  GPA  may  differ 
slightly  or  significantly  from  the  undergraduate  GPAs  posted  on  a  student’s  transcript. 
Dooley  noted  that  the  recalculated  GPA  is  used  to  determine  the  first  digit  of  the  APC 
displayed  in  Figure  1. 

Eligibility  for  all  MBA  resident  degree  programs  except  Financial  Management 
requires  an  APC  Digit  1  at  least  equal  to  “3”  according  to  the  NPS  OR  (2016).  That  is,  for 
admission  to  most  MBA  resident  programs,  a  prospective  student  must  have  a  re¬ 
calculated  minimum  undergraduate  GPA  that  is  greater  than  or  equal  to  2.20  (C+ 
average).  For  the  resident  MBA  in  Financial  Management  and  the  two  DF  EMBA 
programs,  a  re-calculated  undergraduate  GPA  that  is  a  minimum  of  2.60  (B-  average)  is 
required.  This  re-calculated  GPA,  referenced  in  the  NPS  OR  (2016)  Academic  Catalog, 
which  yields  an  APC  Digit  1  that  is  at  least  equal  to  “2,”  imposes  a  stricter  eligibility 
requirement  in  the  form  of  a  higher  grade  point  average  threshold  for  financial 
management  and  the  two  DF  programs. 
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Figure  1.  APC  Determination  of  Digit  1. 

Rules  for  the  First  Digit  (FD)  of  APC  Computation 


FD=5 


GPA  Overall  GPA  for  all  courses 

Source:  Admissions  Office,  unpublished  data  (2014). 

2.  APC  Digit  2:  Mathematics  Background 

APC  digit  2  is  computed  from  undergraduate  Math  courses;  the  required  score  of 
“4”  is  achievable  in  two  ways.  A  student  with  a  minimum  grade  of  “C”  in  at  least  one 
upper-level  Math  course,  including  Linear  Algebra,  receives  a  “4.”  A  student  with  a 
minimum  grade  of  “C”  in  at  least  one  lower  level  Calculus  course,  or  a  combined 
minimum  “B”  average  in  at  least  two  college  level  Math  courses  also  receives  a  “4.” 
Figure  2  explains  computation  of  APC  digit  2.  The  “4”  required  for  APC  Digit  2  is 
consistent  across  GSBPP  Master’s  programs  except  the  DL  Master  of  Science  in  Program 
Management  according  to  the  NPS  OR  (2016).  This  latter  program  requires  an  APC  Digit 
2  of  “3,”  does  not  confer  an  MBA,  and  is  here  excluded. 
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Figure  2.  APC  Determination  of  Digit  2. 

APC  Rules 


Rules  for  the  Second  Digit  (SD)  of  APC  Computation 


Source:  Admissions  Office,  unpublished  data  (2014). 


3.  APC  Digit  3:  Technical/Engineering  Background 

The  third  APC  digit  is  computed  using  the  prospective  student’s  prior  engagement 
with  a  technical,  engineering,  or  physics  program.  Students  with  a  technical-engineering 
or  physics  degree,  or  with  an  average  grade  of  “B”  for  two  calculus-based  physics 
courses,  receive  a  discipline-coded  digit.  All  other  students  are  assigned  a  “5,”  the 
minimum  third  digit  entry  criteria  for  all  GSBPP  resident  and  DL  programs  defined  in  the 
NPS  OR  (2016)  Academic  Catalog.  Prospective  students  who  do  not  meet  any  of  the 
desired  technical,  engineering  or  physics  criteria  are  thus  still  eligible  for  admission, 
explained  in  detail  in  Figure  3. 
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Figure  3.  APC  Determination  of  Digit  3. 


APC  Rules 


Rules  for  the  Third  Digit  (TD)  of  APC  Computation 


Source:  Admissions  Office,  unpublished  data  (2014). 


B.  MATRICULATING  STUDENT  SUCCESS  MEASURES:  TOTAL  NPS, 

MBA  PROGRAM  AND  MBA  CORE  COURSE  GPA  AND  GRADUATION 
RATES 

Matriculating  students’  program  success  is  tracked  by  the  NPS  Registrar’s  Office. 
For  this  project,  student  success  was  assessed  by  total  NPS,  program  and  core  course 
GPA  and  Graduation  Rates.  The  effort  to  understand  “how  one  can  boost  the  value  of 
constructing  knowledge  and  learning  with  ‘others’  in  e-leaming”  (Quintanar  2012,  150) 
has  produced  few  conclusive  findings  on  specific  factors  affecting  program  and  student 
success.  For  this  reason,  data  obtained  from  student  transcripts,  is  analyzed  primarily  to 
provide  a  preliminary  means  of  comparing  resident  and  DL  MBA  student  success  across 
their  program  experience,  as  well  as  at  its  completion.  Importantly,  differences  between 
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the  resident  MBA  and  DL  EMBA  vary  both  by  course  delivery  mode  and  in  the  courses 
required  by  these  programs.  Table  1  displays  these  differences  in  course  requirements  for 
the  resident  and  DL  programs. 


Table  1 .  MBA  Resident  and  EMBA  DL  Core  Course  Load  Comparison. 

Shared  MBA 
Courses  with 
EMBA  DL 

MBA  Resident  Core  Courses _ Program 

GB1000  Quantitative  Skills  for  Graduate  Management  Studies 


GB3010  Managing  for  Organizational  Effectiveness  ✓ 

GB3012  Communications  for  Managers 

GB3014  Ethics  for  Public  Managers 

GB3020  Fundamental  of  Information  Technology 

GB3040  Business  Statistics  &  Data  Analysis 

GB3042  Operations  Management  ✓ 

GB3050  Financial  Reporting  and  Analysis  ✓ 

GB3051  Cost  Management  / 

GB3070  Economics  of  the  Global  Defense  Environment  ✓ 

GB4014  Strategic  Management 

GB4043  Business  Modeling  Analysis  V 

GB4052  Managerial  Finance  / 

GB4053  Defense  Budget  and  Financial  Management  Policy  ✓ 

GB4071  Economic  Analysis  &  Defense  Resource  Allocation 
MN3331,  Systems  Acquisition  and  Program  Management 


EMBA  DL  Core  Courses  not  shared  with  Resident  MBA _ 

GE3011  Management  of  Teams 

GE3221  Principles  of  Acquisition  and  Program  Management  II 
GE3222  Principles  of  Acquisition  and  Program  Management  II 
GE4016  Managing  Strategic  Change 
GE4101  Collaborative  Problem  Solving  1 
GE4102  Collaborative  Problem  Solving  2 
GE4480  Defense  Supply  Chain  Management 


Source:  NPS  Office  of  the  Registrar,  unpublished  data  (2016).MBA  Resident  programs  have 
several  additional  curriculum  specialized  courses.  GB3510  defense  financial  management 
practice  course  is  not  included  in  this  table  is  shared  only  by  the  financial  management 
resident  MBA  and  DL  EMBA  programs  and  is  not  considered  a  core  course.  Selection  of 
core  courses  is  based  on  uniformity,  used  for  calculating  a  core  course  GPA  (CoreQPR). 


To  better  capture  data  reflecting  the  achievement  of  students  in  their  particular 
NPS  MBA  program,  as  well  as  data  allowing  a  comparison  of  student  achievement  across 
programs,  analyses  of  three  grade  point  averages  were  conducted  in  this  MBA  project. 
Two  of  the  grade  point  averages,  TQPR  and  CQPR,  were  pre-calculated  by  the  NPS 
Student  Management  Database.  However,  a  separate  grade  point  average  was  calculated 
based  only  on  student  outcomes  in  the  eight  course  requirements  shared  by  all  NPS  MBA 
programs,  including  Managing  for  Organizational  Effectiveness,  Operations 
Management,  Linancial  Reporting  and  Analysis,  Cost  Management,  Economics  of  the 
Global  Defense  Environment,  Business  Modeling  Analysis,  Managerial  Finance,  and 
Defense  Budget  and  Financial  Management  Policy.  Recalculating  a  CoreQPR  grade  point 
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average  for  these  eight  courses,  which  are  taken  by  both  resident  and  DL  MBA  students, 
permits  comparison  of  a  slightly  more  extensive  and  nuanced  data  sample.  However, 
limits  on  access  to  individual  student  and  course  data  prevented  the  use  of  meaningful 
controls  for  this  portion  of  the  MBA  project  analysis. 

1.  GPAs:  TQPR,  CQPR,  CoreQPR 

NPS  grade  point  averages  were  developed  to  provide  an  indication  of  the  ability 
of  students  to  do  well  in  NPS  programs.  At  NPS,  an  instructor’s  assessment  of  a  student’s 
success  in  meeting  a  course  objective  is  provided  by  a  grade.  To  provide  a  report  of 
student  progress  that  can  be  compared  to  other  NPS  students  and  those  at  other  accredited 
institutions,  the  grade  point  average  is  then  calculated  by  the  NPS  Student  Management 
Database  in  the  NPS  Office  of  Administration  as  follows,  and  recorded  in  each  student’s 
official  transcript. 

The  NPS  Academic  Catalog  defines  the  quarter-hour  value  (course  credit)  for  a 
class  at  NPS  as  “its  scheduled  number  of  weekly  lecture  hours,  plus  one-half  of  the 
scheduled  number  of  laboratory  hours”  (NPS  OR  2016,  16).  To  calculate  each  student’s 
achievement  in  the  class,  its  quarter-hour  value  course  is  multiplied  by  the  point  value  of 
the  student’s  grade,  yielding  a  quality  point  value  for  the  student’s  course  work.  For 
example,  as  explained  in  the  NPS  Academic  Catalog,  “consider  a  student  who  receives  a 
grade  of  ‘B’  in  a  course  that  has  three  hours  of  lecture  and  two  hours  of  lab.  The  quarter- 
hour  value  of  four  quarter-hours  is  multiplied  by  the  point  value  assigned  for  a  grade  of 
‘B’  (3.0);  this  will  result  in  12.0  quality  points  for  the  course.  The  sum  of  the  quality 
points  for  all  courses,  when  divided  by  the  sum  of  the  quarter-hour  credit  of  these 
courses,  provides  a  weighted  evaluation,  a  grade  point  average,  termed  the  Quality  Point 
Rating  (QPR)”  (NPS  OR  2016,  17). 

Grade  point  averages  for  different  courses  of  study,  as  determined  by  a  student’s 
program,  are  reflected  in  this  MBA  project  research  through  the  analysis  of  three  QPR 
measurements.  The  total  course  grade  point  average  (TQPR)  reflects  the  grades  for  all 
courses  a  student  has  taken  at  NPS.  This  total  course  measurement,  for  example,  can 
include  courses  for  certificates  and  other  degree  programs  at  NPS,  in  the  case  of  a  second 
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degree  or  transfer  from  another  degree  program.  Secondly,  the  program  curriculum 
course  grade  point  average  (CQPR)  includes  grades  only  for  those  courses  required  for 
graduation  by  the  student’s  specific  MBA  program.  Lastly,  the  core  course  grade  point 
average  (CoreQPR)  evaluates  grades  for  the  eight  core  courses  required  in  common  by  all 
NPS  MBA  and  EMBA  programs. 

Student  CoreQPR  grade  point  averages  were  calculated  as  follows.  A  qualitative 
analysis  of  the  course  matrix  for  each  resident  MBA  and  EMBA  curriculum  was 
performed  to  identify  core  courses  they  required  in  common;  these  courses  are  reflected 
in  Table  1.  Individual  student  course  and  grade  data  was  then  collected  from  the  NPS 
Student  Management  Database  for  the  sample  population  of  students  who  began  their 
MBA  programs  in  academic  years  2006  through  2013.  Non-core  course  data  was  then 
removed  from  the  individual  student  course  and  grade  data.  Core  QPRs  for  the  MBA 
curricula  were  calculated  in  accordance  with  the  NPS  Academic  Catalog  and  Policy 
Manual,  by  dividing  the  summation  of  calculated  core  course  quality  points  by  the 
summation  of  core  course  units. 

2.  Graduation  Rates 

Graduation  rates  provide  another  factor  used  in  the  current  study,  and  are 
calculated  based  on  the  expected  date  of  MBA  program  completion  for  each  NPS  student. 
Student  who  graduate  on  or  before  their  expected  graduation  date  are  considered  to  be  an 
on-time  graduate.  Those  students  who  graduate  after  their  original  expected  graduation 
date  are  considered  to  be  a  late  graduate.  Both  on-time  and  late  graduates  are  deemed 
graduates.  Students  who  have  not  yet  graduated  due  to  thesis  extension  or  will  not 
graduate  due  to  attrition  are  here  considered  a  non-grad. 

The  data  provided  by  NPS  MBA  student  transcripts  including  the  different  grade 
point  averages  (QPR)  and  program  completion  dates  (graduation  rates)  are  analyzed  in 
this  project  to  leam  about  the  differences  between  resident  and  DL  programs.  In  addition 
to  analyzing  student  success  outcomes  provided  by  student  transcripts,  more  qualitative 
data  including  student  perceptions  of  their  program  experience  is  also  analyzed  for 
comparisons  between  resident  and  DL  MBA  programs. 
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C.  GRADUATING  STUDENT  SURVEY:  STUDENT  ACADEMIC 

EXPERIENCE  MEASURE 

Lastly,  to  provide  an  understanding  of  student  success  from  the  point  of  view  of 
the  students  themselves,  I  performed  a  qualitative  analysis  of  three  key  questions  within 
the  Graduating  Student  Survey  (GSS).  In  addition  I  completed  a  response  rate  analysis. 
The  GSS  provides  submitted  responses  by  students  upon  completion  of  their  degree 
programs.  For  this  project  I  collected  and  reviewed  GSS  Responses  for  GSBPP  students 
for  the  graduation  academic  years  of  2010  through  2015. 

In  order  to  observe  between-group  differences  for  resident  and  DL  students  in  the 
NPS  MBA  programs,  four  areas  were  assessed:  entry  requirements  (APC),  grade  point 
averages  (TQPR,  CQPR,  Core  QPR),  graduation  status  (on-time  graduation,  late 
graduation,  did  not  graduate),  and  online  responses  to  three  survey  questions  assessing 
student  NPS  MBA  program  experience.  I  evaluated  the  findings,  without  controls,  using 
different  analyses.  These  analyses  were  conducted  to  contribute  to  greater  awareness  of 
between-group  differences  that  might  potentially  inform  more  comprehensive  studies 
aimed  at  improving  Resident  and  DL  MBA  program  quality. 
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V.  COMPARING  STUDENT  OUTCOMES  ACROSS  DELIVERY 

MODES 


After  collecting  resident  and  DL  MBA  student  data  included  in  this  MBA  project 
research,  I  performed  analyses  that  evaluated  the  following  factors.  First  were  NPS  entry 
requirements  (APC)  for  MBA  and  EMBA  students.  I  also  included  an  examination  of 
student  grade  point  averages,  grouped  to  reflect  all  NPS  courses  taken  during 
matriculation  for  the  MBA  or  the  EMBA  (TQPR),  all  MBA  or  EMBA  program- specific 
courses  (CQPR),  and  all  core  courses  required  both  by  the  MBA  and  the  EMBA 
programs  (CoreQPR).  I  also  examined  graduation  rates  according  to  graduation  status 
(on-time  graduates,  late  graduates,  graduates,  and  non-graduates).  These  analyses  were 
conducted  using  t-tests  of  unequal  variances,  correlation  tests  and  a  mean  analysis.  All 
analyses  were  performed  to  identify  factors  that  may  improve  understanding  of  the 
potential  impact  on  NPS  MBA  student  success  of  preparedness,  as  measured  by  program 
entry  requirements,  and  student  experiences  of  quality  and  support  during  matriculation, 
as  reported  in  the  GSS. 

These  analyses  were  conducted  to  evaluate  between-group  differences  in  resident 
and  DL  MBA  programs.  These  analyses  did  not  use  the  preferred  research  analysis 
method  of  propensity  score  matching  nor  control  for  well-known  and  important 
influences  on  student  outcomes.  Such  influences  may  include  differences  that  exist  in 
syllabi,  teaching  quality,  testing  methods,  learning  approaches  and  other  known 
contributors  or  inhibitors  of  student  success.  Nevertheless  the  present  data,  obtained 
through  analyses  of  overall  student  success  measures  for  resident  and  DL  MBA  students 
at  NPS,  do  offer  preliminary  results  on  specific  differences  between  the  two  delivery 
modes,  bringing  attention  to  areas  that  may  need  improvement  in  the  defense  focused 
NPS  MBA  programs. 

A.  T-TESTS 

To  analyze  NPS  entry  requirements  (APC)  and  the  three  types  of  grade  point 
average,  as  well  as  grades  from  the  eight  individual  core  courses  required  by  both  the 
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resident  and  DL  programs,  I  used  t-tests  to  observe  any  between-group  differences  in 
means  that  proved  significant  at  the  95%  confidence  level.  Differences  between  the  two 
delivery  modes  were  observed  in  t-test  and  mean  results  for  the  three  MBA  student 
sample  sets.  These  three  sets  included:  civilian  and  military  students,  military  students 
only,  and  students  grouped  by  gender  for  all  student  success  factors,  except  their 
curriculum  course  grade  point  averages.  These  mean  and  t-test  analyses  of  NPS  MBA 
student  success  factors  yield  the  outcomes  shown  in  Table  2. 


Table  2.  Summary  Statistics  of  Student  Success  Factors:  APCs  and  GPAs. 


Resident(Res) 

Distance  learning(Dt ) 

Res/Dl 

Reject 

Null  HO 

Justification 

for  Research 

Cannot  Reject 
Null  Ho- No 

Justification 

for  Research 

Mean 

SD 

Mean 

SD 

Diff 

Significant 

Insignificant 

APCs 

Civilian  and  Military  Combined  (ALL) 

211.6 

105 

188.5 

122.5 

2.54L-05 

✓ 

Military  (AIL) 

212.6 

104.1 

183.3 

121.1 

1.17E-06 

✓ 

Civilian  and  Military  Combined  (f  emale) 

191.2 

109.5 

194.9 

125.8 

0.824 

✓ 

Civilian  and  Military  Combined  (Male) 

213.6 

103.9 

187.5 

122.1 

7.49F-06 

✓ 

Military  (Female) 

195.4 

109.7 

172.9 

135.4 

0.339 

✓ 

Military  (Male) 

214.3 

103.4 

184.2 

120.2 

1.37E-06 

✓ 

TQPR 

Civilian  and  Military  Combined  (All) 

3.71 

0.22 

3.72 

0.43 

0.671 

✓ 

Military  (ALL) 

3.71 

0.22 

3.73 

0.41 

0.424 

✓ 

Civilian  and  Military  Combined  (Female) 

3.68 

0.24 

3.72 

0.20 

0.329 

✓ 

Civilian  and  Military  Combined  (Male) 

3.71 

0.22 

3.72 

0.45 

0.822 

✓ 

Military  (Female) 

3.68 

0.24 

3.72 

0.21 

0.346 

✓ 

Military  (Male) 

3.71 

0.22 

3.73 

0.22 

0.501 

✓ 

CoreQPR 

Civilian  and  Military  Combined  (ALL) 

3.6S 

0.47 

3.73 

0.42 

4.25F  29 

✓ 

Military  (All) 

3.65 

0.47 

3.74 

0.41 

2.80f-30  — 

✓ 

Civilian  and  Military  Combined  (Female) 

3.61 

0.51 

3.67 

0.48 

0.021  • 

V 

Civilian  and  Military  Combined  (Male) 

3.65 

0.47 

3.74 

0.40 

6.11E-31  •" 

✓ 

Military  (Female) 

3.61 

0.52 

3.72 

0.48 

0.046  * 

✓ 

Military  (Male) 

3.71 

0.47 

3.73 

0.40 

5.72F-30  ••• 

✓ 

Source:  See  text  for  data  collection  methods  and  details.  *  p<.05,  **  p<.01,  ***  p<.001.  AY2009,  APC 
Sample  Size:  NPS  Admissions  created  the  Admissions  Data  Management  System  (AMS)  for  students 
seeking  admissions  to  NPS  programs.  All  APCs  are  calculated  manually,  but  prior  to  AY2009  APC 
scores  for  entering  students  were  not  all  manually  entered  into  this  system.  A  total  of  61  who  enrolled 
prior  to  AY2009  are  missing  APC  scores. 


1.  NPS  Entry  Requirements:  APCs 

For  the  entry  criteria  variable  (APC),  the  total  sample  size  for  all  MBA  civilian 
and  military  students  was  N=l,747  students,  and  for  military  students  only,  the  sample 
size  was  N=l,473  students.  For  these  populations,  t-test  findings  indicated  differences  to 
be  somewhat  significant,  at  p=2.54E-05  and  p=1.17E-06,  respectively.  A  similar  finding 
was  observed  for  the  larger  all  male  (N=  1,549)  and  the  smaller  military-only  male 
(N=l,349)  populations,  p=7.49E-06  and  p=1.37  E-06.  By  contrast,  the  results  were 
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insignificant  for  female  students,  both  when  analyzed  by  the  larger  sample  of  civilian- 
and-military  females  (N=198),  at  p=0.824  and  for  the  smaller  military-only  female 
sample  (N=124),  at  p=0.339.  This  research  finding  of  a  gender  difference  in  student 
outcomes  by  entry  criteria  for  the  all-inclusive  male  group  and  the  military-only  male 
group  may  merit  further  study  and  clarification  to  understand  factors  associated  with 
gender  that  have  the  potential  to  influence  student  success. 

2.  Student  Evaluation:  Grade  Point  Averages 

For  the  two  grade  point  average  variables  analyzed  by  t-test,  all  NPS  courses 
(TQPR)  and  all  core  courses  (Core  QPR),  the  total  sample  size  for  both  MBA  and  EMBA 
civilian-and-military  students  was  N=l,808.  A  very  strong  between-group  correlation 
(.99)  was  noted  for  grade  point  averages  (TQPR  and  CQPR),  justifying  the 
interchangeable  use  of  one  of  the  two  QPR  values  to  simplify  analysis  (for  a  more 
detailed  discussion  see  Correlation  Analysis).  Of  this  total  sample  size  (N=l,808),  a  total 
of  917  students  were  enrolled  in  the  seven  resident  MBA  programs,  with  a  total  of  891 
students  enrolled  in  the  two  DL  EMBA  programs.  Overall,  no  between-group  differences 
were  noted  in  results  for  the  civilian-and-military  and  the  military-only  student  sample 
sets  for  TQPR.  Similarly,  T-tests  for  student  grade  point  averages  for  all  courses 
completed  (TQPR),  as  well  as  for  both  gender  samples,  showed  no  significant  between- 
group  differences. 

However,  the  grade  point  averages  for  core  courses  shared  by  the  MBA  and 
EMBA  programs  (CoreQPR)  did  point  to  a  somewhat  significant  between-group 
difference  in  both  the  overall  sample  and  by  gender  differentiate  sample  populations. 
These  findings  suggest  that  further  investigation  is  warranted  to  identify  specific  factors 
that  may  influence  student  outcomes,  as  indicated  by  grade,  for  the  eight  core  courses 
common  to  the  MBA  and  EMBA  programs.  Table  3  displays  these  core  course  outcomes. 
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Table  3.  Summary  Statistics  of  Student  MBA  Core  Course  Level  Grades. 


Reject  Cannot  Reject 

Null  Ho  *  Null  Ho=No 
Justification  Justification 


Residenl(Res) 

Distance  LcarniriK(DL) 

Re*/Dl 

foe  Research 

for  Research 

Mean 

SO 

Mean 

SD 

Diff 

Significant 

InsiRnificant 

Core  MIIA  Course  erodes 

GB3042  Rfft/GI  3042  Ol 

3.65 

0.44 

3.76 

0.39 

7.931  *06 

✓ 

GB3051  R#*/  GE  3051  DL 

3.60 

0.42 

3.41 

0.61 

0.0013  *• 

✓ 

GB30/0  Kes/  GL30/0  DL 

3.64 

0.39 

3.56 

0.52 

0.0015  *• 

✓ 

GB4052  Re*/GE4052  DL 

3.71 

0.54 

3.80 

0.40 

9.70E-05 

✓ 

GB4053  Res/GE4053  DL 

3.77 

0.25 

3.64 

0.36 

4.52L-15 

✓ 

GB3010  Re*/  CL 3010  DL 

3.B5 

0.21 

3.87 

0.34 

0.22 

✓ 

GB3050  Re*/G£3050  DL 

3.69 

0.51 

3.73 

0.44 

0.17 

✓ 

G 134043  Re*/  GL4043  DL 

3.68 

0.52 

3.73 

0.43 

0.078 

Source:  See  text  for  data  collection  methods  and  details.  *  p<.05,  **  p<.01,  ***  p<.001. 

3.  Student  Evaluation:  MBA  Individual  Core  Course  Grades 

I  conducted  a  preliminary  analysis  of  core  course  outcomes  in  an  effort  to  help 
identify  factors  that  may  drive  between-group  differences  observed  for  resident  and  DL 
student  grade  point  average  for  core  courses  common  to  both  programs  (CoreQPR). 
Through  further  examination  of  significant  findings  by  each  specific  core  course,  I 
identified  five  courses  among  the  eight  core  courses  that  drove  the  statistically  significant 
finding  of  between-group  differences  in  CoreQPR.  These  courses  were  GB3042 
Operations  Management,  GB3051  Cost  Management,  GB3070  Economics  of  the  Global 
Defense  Environment,  GB4052  Managerial  Finance,  and  GB  4053  Defense  Budget  and 
Financial  Management  Policy. 

The  three  other  courses,  GB3010  Managing  for  Organizational  Effectiveness, 
GB3050  Financial  Reporting  and  Analysis,  and  GB4043  Business  Modeling  Analysis, 
yielded  insignificant  outcomes.  Of  these  three,  GB3050,  an  introductory  accounting 
course  that  is  mathematically  intensive,  was  a  conspicuous  outlier,  particularly  given 
findings  in  the  literature  that  would  suggest  the  likelihood  of  a  statistically  significant 
between-group  difference  for  this  class  (Anstine  and  Skidmore  2005;  Ni  2013).  In  this 
project’s  findings,  no  such  difference  was  observed. 

Also  in  contrast  to  findings  in  the  literature,  which  suggest  that  courses  having  a 

weaker  mathematics  component  are  unlikely  to  show  any  between-group  significant 

difference  (Harmon,  Alpert,  and  Lambrinos  2014;  Gibson  2008),  GB3010,  an 

introductory  organizational  management  course,  shows  a  statistically  significant 
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difference.  Moreover,  for  the  majority  of  core  courses  required  by  both  the  resident  and 
DL  programs,  some  statistical  significance  appears  to  exist.  These  findings  may  indicate 
other  potential  areas  for  more  comprehensive  course  level  research. 

B.  CORRELATION  ANALYSIS 

I  conducted  a  correlation  analysis  to  establish  between-group  relationships  for 
factors  evaluated  in  this  MBA  project  research  that  might  benefit  from  a  closer 
examination  in  future  studies.  These  correlation  analyses,  conducted  only  as  a 
preliminary  inquiry,  did  not  apply  controls  for  influential  factors,  as  would  be  required  to 
obtain  more  conclusive  findings.  Key  findings  from  these  studies  are  first  summarized, 
with  the  specific  methods  used  to  obtain  them  then  provided. 

A  very  strong  between-group  correlation  (.99)  was  noted  for  the  grade  point 
averages  TQPR  and  CQPR  justifying;  additionally,  both  TQPR  and  CQPR  correlated 
strongly  (.81)  to  MBA  core  course  grade  point  averages  (CoreQPR).  Thus,  in  this  study 
the  grade  point  averages  reflected  in  the  student  CoreQPR  can  be  considered  a  reasonable 
proxy  for  all  three  grade  point  averages  in  this  study.  By  contrast,  the  correlation 
coefficient  of  .55  for  military  and  civilian  TQPRs  shows  only  a  moderate  between-group 
relationship,  suggesting  that  the  use  of  one  sample  population  over  the  other  may  be 
inadvisable. 

The  initial  correlation  analysis  completed  for  this  MBA  project  research  examined 
admission  criteria  (APC)  and  the  three  grade  point  averages  (TQPR,  CQPR,  and 
CoreQPR).  To  make  it  possible  to  perform  correlation  analyses  for  the  factors  on-time 
graduation,  waivers  of  admission  criteria  (APC)  and  grade  point  average  (QPR),  binary 
numeral  conversions  were  accomplished  by  an  identical  methodology  for  all  factors. 
Firstly,  the  factors  were  converted  to  a  Y-  N  binary  option,  with  each  option  then  coded 
either  as  Y=0  or  N=1  for  subsequent  analysis. 

The  correlation  coefficient  results  yielded  from  this  process  are  shown  in  Table  4. 
These  results  indicate  a  weak  negative  correlation  between  admission  criteria  (APC)  and 
each  of  the  three  grade  point  averages  (TQPR,  CQPR,  CoreQPR).  This  finding  is 
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important  to  note,  as  it  indicates  that  no  obvious  relationship  exists  between  students’ 
academic  criteria  score  (APC)  and  success  in  their  MBA  or  EMBA  program. 


Table  4.  APC  Correlation  Coefficient  Table:  APCs  versus  GPAs  (QPRs). 


APC 

TQPR 

CQPR 

CoreQPR 

Civilian  &  Military 

-0.12 

-0.12 

-0.23 

Military  Only 

-0.13 

-0.13 

-0.23 

Source:  See  text  for  data  collection  methods  and  details. 


Students  who  are  seeking  admission  into  a  NPS  MBA  resident  or  DL  program, 
but  who  do  not  meet  NPS  admissions  criteria  for  any  one  of  the  APC  digits,  may  receive 
an  APC  waiver  from  the  program  department  administration  (Dooley,  personal 
communication,  2017).  I  therefore  further  evaluated  admissions  criteria  (APC)  waivers  so 
as  to  identify  any  relationship  between  these  waivers  and  the  three  graduation  types  that 
might  indicate  the  potential  impact  of  a  waiver  on  students’  successful  program 
completion.  Table  5  displays  the  percentage  of  waivers  required  by  Military  and  Civilian 
student  populations  who  began  their  MBA  or  EMBA  programs  in  AY2006-AY2013. 


Table  5.  Percentage  of  APC  Waivers  by  Start  AY  and  Student  Type. 


Sturt  A 

Y2006 

Sturt  AY200/ 

Sturt  AY2008 

Sturt  A 

Y2U09 

Sturt  A 

Y2010 

Delivery  Mode 
Res 

Student  Type 

Student  Count 

%  of  Waivers 

22% 

Student  Count 
86 

%  of  Waivers 

2/% 

Student  Count 
100 

%  of  Waivers 

1/% 

Student  Count 
/9 

%  of  Waivers 

24% 

Student  Count 
114 

%  of  Waivers 

16% 

Military 

14/ 

Civilian 

16 

19% 

4 

25% 

7 

43% 

9 

33% 

8 

38% 

Res  lotal 

Res 

164 

21% 

90 

2 /% 

10/ 

19% 

88 

25% 

122 

1/% 

Dl 

Military 

5? 

6% 

78 

23% 

93 

22% 

98 

20% 

79 

34% 

Civilian 

2 

0% 

21 

29% 

14 

21% 

34 

50% 

49 

53% 

Dl  lotal 

Dl 

54 

6% 

99 

24% 

10/ 

21% 

132 

28% 

128 

41% 

Grand  Total 

ALL 

218 

17% 

189 

25% 

214 

20% 

220 

27% 

250 

30% 

Start  AY20 11 

Start  AY2012 

Start  AY2013 

Total  AY2006-13 

Delivery  Mode 

Student  type 

Student  Count 

%  ol  Waivers 

Student  Count 

%  ol  Waivers 

Student  Count 

%  ol  Waivers 

Student  Count 

%  ol  Waivers 

Res 

Military 

88 

20% 

109 

16% 

120 

14% 

843 

19% 

Civilian 

8 

13% 

6 

17% 

15 

7% 

73 

22% 

Ras  Total 

Res 

96 

20% 

116 

16% 

138 

13% 

921 

19% 

DL 

Military 

108 

22% 

98 

24% 

72 

22% 

678 

22% 

Civilian 

24 

t>0% 

35 

23% 

34 

38% 

213 

40% 

DL  Total 

DL 

132 

27% 

133 

24% 

100 

27% 

891 

27% 

Grand  total 

ALL 

228 

24% 

249 

20% 

244 

19% 

1812 

23% 

Source:  See  text  for  data  collection  methods  and  details.  Appendix  Table  12  delineates  APC  waivers  by 
Military  Service. 


Table  6  displays  the  correlation  of  APC  waivers  to  three  of  the  graduation 
categories  (on-time  graduate,  graduate,  non- graduate).  Resident  and  DL  students  who 
required  waivers,  whether  in  the  civilian- and-military  or  the  military-only  sample,  have  a 
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weak  negative  correlation  to  the  on-time  graduated  and  graduated  categories.  Resident 
and  DL  students  who  did  not  graduate  have  a  weak  positive  correlation  to  waivers  of 
admissions  criteria  (APC).  These  marginal  relationships  loosely  coincide  with  other 
research,  such  as  the  study  by  Christensen  and  Nance  (2012),  who  found  that  students  not 
meeting  entry  criteria  might  have  higher  dropout  rates.  Further  exploration  of  the 
relationships  between  entry  criteria  and  graduation  may  thus  be  warranted. 


Table  6.  APC  Waiver  versus  Graduation  Type  by  Student 
Population  Type  and  Res/DL. 


On-Time  Res 

On-Time  DL  | 

Grad-Res 

Grad-DL 

Non-Grad  Res 

Non-Grad  DL  | 

Waiver(APC-Entry  Criteria  Not  Met)  ALL 

-0.094 

-0.075 

-0.096 

-0.116 

0.087 

0.116 

Waiver(APC-Entry  Criteria  Not  Met)  MIL 

-0.122 

-0.077 

-0.071 

-0.140 

0.073 

0.140 

Source:  See  text  for  data  collection  methods  and  details.  On-Time  is  defined  as  students  who  graduate  at  the 
time  expected  without  an  extension.  Grad  is  defined  as  students  who  graduated  within  or  beyond  the 
expected  graduation  date,  with  or  without  extensions.  Non-Grad  is  defined  as  a  student  who  has  yet  to 
graduate. 

A  correlation  analysis  of  admissions  criteria  (APC)  waivers  to  CoreQPR  grade 
point  averages  was  conducted  and  is  displayed  in  Table  7.  For  DL  EMBA  students,  a 
weak  positive  correlation  was  observed  between  student  grade  point  averages  (CoreQPR) 
equal  to  a  “C”  or  lower  and  entry  criteria  (APC)  waivers.  Similarly,  a  weak  positive 
correlation  was  noted  for  resident  MBA  students  who  required  waivers  and  for  those  who 
obtained  a  failing  grade  point  average.  These  results  may  provide  additional  support  for 
the  findings  of  Christensen  and  Nance  (2012),  who  report  that  students  with  lower 
undergraduate  GPAs  are  also  less  likely  to  be  successful  in  their  MBA  programs. 


Table  7.  APC  Waiver  versus  Core  QPR  by  Student  Population  Sample  and 

Res/DL. 


ALL  Res  Waivers 

ALL  DL  Waivers 

MIL  ONLY  Res  Waivers 

MIL  ONLY  DL  Waivers 

A (3.7-4) 

-0.142 

-0.162 

-0.124 

-0.191 

B (2.7-3.69) 

0.117 

0.139 

0.096 

0.166 

C  (1.7-2.69) 

-0.020 

0.126 

-0.021 

0.144 

D  (1-1.69) 

n/a 

n/a 

n/a 

n/a 

F  (<1) 

0.051 

n/a 

0.054 

n/a 

Source:  See  text  for  data  collection  methods  and  details. 
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In  a  correlation  analysis  of  graduation  types  to  CoreQPR  grade  point  averages, 
displayed  in  Table  8,  no  clear  trend  can  be  seen.  A  CoreQPR  grade  point  average  of  “A” 
shows  a  weak  positive  correlation  to  both  resident  and  DL  graduation  categories. 
However,  for  both  these  categories,  a  weak  negative  correlation  exists  with  a  CoreQPR 
grade  point  average  of  “C.”  No  conclusive  correlation  can  be  observed  between 
CoreQPRs  grade  point  averages  of  “B,”  “D,”  and  “F”  and  student  graduation  categories. 
This  preliminary  correlation  analysis  thus  does  not  support  a  relationship  between  student 
grade  point  averages  in  resident  and  DL  MBA  commonly  required  core  courses  and  the 
three  student  graduation  categories  evaluated. 


Table  8.  CoreQPR  Correlation  Coefficient  Table: 

CoreQPR  versus  Graduation  Type. 


Military  Students  GPA 

On-Time  Res 

On-Time  DL 

Grad-Res 

Grad-DL 

Non-Grad  Res 

Non-Grad  DL 

A  (3.7-4) 

0.064 

0.122 

0.175 

0.463 

0.183 

0.023 

B  (2.7-3.69) 

0.010 

-0.042 

-0.165 

-0.427 

-0.229 

-0.017 

C  (1.7-2.69) 

-0.123 

-0.225 

-0.227 

-0.267 

-0.010 

-0.015 

D  (1-1.69) 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

F(<1) 

-0.137 

n/a 

0.009 

n/a 

-0.017 

n/a 

Source:  See  text  for  data  collection  methods  and  details.  On-Time  is  defined  as  students  who  graduate  at  the 
time  expected  without  an  extension.  Grad  is  defined  as  students  who  graduated  within  or  beyond  the 
expected  graduation  date,  with  or  without  extensions.  Non-Grad  is  defined  as  a  student  who  has  yet  to 
graduate. 

In  summary,  no  obvious  relationship  was  seen  from  the  correlation  analysis 
performed  between  student  entry  criteria  scores  and  the  three  grade  point  averages. 
However,  some  weak  relationships  were  observed  between  the  binary  factors  of  APC 
waivers  and  DL-student  lower  CoreQPR  grade  point  averages.  Another  weak  relationship 
was  found  between  APC  waivers  and  students  who  do  not  graduate.  Taken  together, 
these  findings  offer  some  support  for  earlier  research  that  DL  MBA  students  who  do  not 
meet  APC  admissions  criteria  are  at  risk  of  performing  poorly.  However,  for  both 
resident  and  DL  MBA  students,  mixed  findings  were  observed  for  a  relationship  between 
CoreQPR  grade  point  average  ranges  and  the  three  graduation  types.  The  variations 
observed  in  these  results  motivated  me  to  further  analyze  the  three  different  grade  point 
averages  by  performing  a  mean  analysis. 


46 


c. 


MEAN  ANALYSIS 


The  mean  analysis  of  grade  point  averages  was  calculated  by  averaging  individual 
student  GPAs  provided  by  the  three  grade  point  average  types  (TQPR,  CQPR, 
CoreQPR).  An  additional  analysis  evaluated  these  factors  by  program  instruction  mode 
(Resident  and  DL),  civilian-and-military  and  military-only  student  status  and  academic 
year  program  start  date  (2006  through  2013).  Results  obtained  from  this  analysis  are 
displayed  in  Table  9. 


Table  9.  MBA  Mean  Analysis  by  GPA  (QPR)  Type  and  Student  Sample  Type 

(Civilian-and-Military  and  Military-Only). 


Civilian  &  Military_ Military  ONLY 


Resident_ Distance  Learning  (DL) Resident_ Distance  Learning  (DL) 


TQPR 

CQPR 

CoreQPR 

TQPR 

CQPR 

CoreQPR 

TQPR 

CQPR 

CoreQPR 

TQPR 

CQPR 

CoreQPR 

2006 

3.72 

3.72 

3.67 

3.66 

3.66 

3.63 

3.73 

3.72 

3.68 

3.66 

3.66 

3.63 

2007 

3.69 

3.68 

3.64 

3.67 

3.67 

3.68 

3.69 

3.68 

3.63 

3.67 

3.67 

3.68 

2008 

3.65 

3.65 

3.61 

3.57 

3.57 

3.71 

3.65 

3.65 

3.61 

3.64 

3.64 

3.71 

2009 

3.66 

3.66 

3.56 

3.73 

3.73 

3.70 

3.64 

3.65 

3.54 

3.72 

3.72 

3.68 

2010 

3.72 

3.72 

3.69 

3.75 

3.75 

3.71 

3.73 

3.74 

3.71 

3.77 

3.77 

3.70 

2011 

3.72 

3.72 

3.65 

3.77 

3.77 

3.77 

3.71 

3.71 

3.64 

3.81 

3.81 

3.83 

2012 

3.75 

3.75 

3.69 

3.72 

3.72 

3.75 

3.75 

3.75 

3.69 

3.71 

3.71 

3.75 

2013 

3.75 

3.76 

3.71 

3.78 

3.79 

3.80 

3.76 

3.77 

3.72 

3.77 

3.78 

3.81 

All  Years 

3.71 

3.71 

3.66 

3.72 

3.72 

3.73 

3.71 

3.71 

3.66 

3.73 

3.73 

3.73 

Source:  See  text  for  data  collection  methods  and  details.  Includes  0.0  QPRs  for  Non-Grads. 


For  the  cumulative  academic  years  2006-2013,  the  mean  grade  point  averages  for 
all  three  QPRs  observed  (TQPR,  CQPR,  CoreQPR)  in  both  the  military-and-civilian  and 
military-only  populations,  DL  QPRs  are  slightly  higher  than  those  of  resident  students. 
Similarly,  for  students  who  began  their  program  in  AY2010,  all  three  QPRs  are  slightly 
higher  for  DL  than  for  resident  students.  In  addition,  for  students  starting  in  AY2007, 
2008,  and  2012,  CoreQPRs  are  higher  for  DL  than  for  resident  students.  These  CoreQPR 
findings  differ  slightly  from  other  researchers’  results  for  TQPR  among  NPS  students 
across  all  programs. 

In  contrast,  in  AY2006,  when  examining  student  grade  point  averages  for  each 
separate  year,  the  means  for  all  three  grade  point  averages  (QPR)  are  slightly  higher  for 
resident  than  for  DL  students.  Moreover,  in  start  years  AY2007,  2008  and  2012,  both  the 
TQPR  and  CQPR  resident  means  are  slightly  higher  than  those  of  DL  students.  These 
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results  are  similar  to  the  findings  in  previous  NPS  studies  by  Fodor  (2016)  and  Bacolod 
and  Chaudhary  (2016).  Taken  together  these  results  do  not  consistently  confirm  nor  differ 
with  the  findings  in  previous  studies,  suggesting  that  a  more  comprehensive  analysis 
might  be  useful. 

D.  GRADUATION  RATE  DATA 

Three  of  the  graduation  categories  (on-time  graduate,  late  graduate,  non-graduate) 
were  evaluated  to  help  substantiate  other  factors  contributing  to  or  impeding  successful 
program  completion.  These  graduation  rate  data  were  examined  to  determine  between 
group  differences  in  outcomes  for  resident  MBA  and  DL  EMBA  students.  Graduation  is 
a  primary  student  success  factor  evaluated  in  this  project,  with  the  three  grade  point 
averages  (QPR)  also  considered  of  particular  importance.  Figure  4  displays  graduation 
rate  data. 


Figure  4.  MBA  Graduation  Type  by  Student  Population  Sample,  and  Res/DL, 
Cumulative  Start  Academic  Years  2006-2013. 


Source:  See  text  for  data  collection  methods  and  details.  Graduate  data  collected  displays  the 
proportions  for  three  of  the  graduate  categories.  On-Time  is  defined  as  students  who 
graduate  at  the  time  expected  without  an  extension.  Late  is  defined  as  students  who  graduate 
beyond  the  expected  graduation  date,  with  extensions.  Non-Grad  is  defined  as  a  student  who 
has  yet  to  graduate.  Appendix  Figures  5  and  6  provide  delineation  by  academic  start  year. 


All  start  academic  years  show  that  DL  students  have  an  almost  5%  lower  on-time 
graduation  rate  than  that  of  resident  students,  and  an  almost  8%  lower  overall  graduation 
rate.  The  finding  in  this  smaller  sample  of  MBA  students  supports  findings  of  Bacolod 
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and  Chaudhary  (2016)  for  NPS  students  across  all  programs  in  the  same  years,  AY2006- 
AY2013.  Bacolod  and  Chaudhary  (2016)  report  that  DL  students  across  all  programs  are 
15%  less  likely  to  graduate  than  their  resident  counterparts.  However,  in  the  MBA-only 
cohorts  examined  in  this  project,  DL  MBA  students  fared  better  than  NPS  DL  students 
across  all  programs  in  terms  of  successful  program  completion  (Bacolod  and  Chaudhary 
2016). 

In  line  with  overarching  study  goals  discussed  by  Ni  (2013),  I  considered  a 
comprehensive  evaluation  of  a  variety  of  measures  to  be  important  for  identifying  key 
factors  that  support  student  success.  Particularly  after  these  project  analyses  revealed 
mixed  results  for  many  measures  obtained  from  student  records,  I  became  motivated  to 
expand  my  examination  to  include  student  viewpoints  about  their  MBA  programs  and 
academic  experiences.  I  thus  attempted  in  the  following  section  to  address  discontinuity 
of  project  findings  by  including  student  responses  to  those  Graduating  Student  Survey 
(GSS)  questions  that  address  student  perceptions  of  their  program  experience. 

E.  ACADEMIC  EXPERIENCE  MEASURE:  GRADUATING  STUDENT 

SURVEY 

For  this  MBA  project,  I  performed  a  qualitative  analysis  of  Graduating  Student 
Survey  (GSS)  responses  to  three  key  questions  addressing  resident  and  DL  student 
perceptions  of  their  MBA  academic  experience  at  NPS.  For  this  analysis  I  performed  a 
response  rate  analysis  and  examined  GSS  responses  for  the  graduation  academic  years 
2010  through  2015. 

The  GSS  survey  instrument  collects  responses  submitted  by  students  upon 
completion  of  their  degree  programs.  Through  a  total  of  37  questions,  the  survey  provides 
a  mechanism  for  students  to  rate  their  NPS  programs  and  experience  on  a  five  point 
Likert  scale  (low  to  high).  Three  questions  (Q6,  Qll,  Q31)  assess  students’  perception  of 
their  program  and  opinion  of  their  program’s  success.  For  this  segment  of  my  MBA 
project  research,  I  collected  a  total  of  1,005  accumulated  GSS  responses  from  GSBPP 
degree  students  who  graduated  in  academic  years  AY2010-AY2015.  Of  these  responses, 
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762  were  from  students  enrolled  in  the  eight  specialized  resident  MBA  programs  and  the 
two  EMBA  programs. 

For  the  graduation  years  included  in  for  analysis,  more  than  half  of  all  NPS 
GSBPP  degree  program  civilian  and  military  graduate  students  responded.  Sheehan 
(2001),  who  evaluated  response  rates  for  different  survey  delivery  modes,  reports  that 
many  factors  influence  email  solicited  survey  response  rates,  including  the  year  it  is 
solicited.  University  of  Texas  (n.d.)  stated  that  the  standard  percentage  for  a  “good” 
response  rate  for  email  surveys  is  greater  than  or  equal  to  50%.  Therefore,  for  the  years 
examined  in  this  project,  NPS  GSBPP  students  have  “good”  survey  response  rates  for  the 
email  solicited  GSS  instrument.  When  observing  military-only  students  enrolled  in  all 
GSBPP  degree  programs,  the  response  rate  increased  to  59%,  as  reflected  in  Table  10.  By 
contrast,  GSBPP  DL  students  have  a  much  lower  response  rate,  irrespective  of  program; 
those  who  do  respond,  report  a  more  positive  opinion  of  their  program  experience  than 
resident  students.  When  observing  only  GSBPP  MBA  degree  programs,  whether  resident 
or  DL,  the  response  rate  decreases  slightly  for  most  years,  indicating  a  slightly  lower 
willingness  of  MBA  students  to  participate  in  the  survey. 


Table  10.  Graduating  Student  Survey  (GSS)  Response  Rates  for  Students 

Graduating  AY2010-AY2015  by  Year. 


Civilian  &  Military 

Military  ONLY 

Resident 

DL 

All 

Resident 

DL 

ALL 

GSBPP  All  Students 

2010 

78% 

44% 

57% 

75% 

22% 

60% 

2011 

84% 

n/a 

37% 

84% 

n/a 

46% 

2012 

73% 

39% 

39% 

74% 

6% 

47% 

2013 

81% 

54% 

69% 

80% 

57% 

71% 

2014 

60% 

25% 

53% 

60% 

44% 

62% 

2015 

76% 

21% 

73% 

73% 

54% 

65% 

All  Years 

75% 

51% 

55% 

74% 

30% 

59% 

MBA  Only  Students 

2010 

86% 

23% 

52% 

83% 

26% 

59% 

2011 

92% 

n/a 

36% 

92% 

n/a 

43% 

2012 

77% 

0% 

40% 

77% 

0% 

46% 

2013 

88% 

52% 

67% 

88% 

55% 

70% 

2014 

64% 

59% 

52% 

65% 

59% 

63% 

2015 

75% 

48% 

65% 

71% 

51% 

63% 

All  Years 

79% 

30% 

52% 

78% 

32% 

57% 

Source:  See  text  for  data  collection  methods  and  details.  *201 1  Distance  Learning 
(DL)  data  is  not  available. 
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For  the  scope  of  this  project  three  relevant  questions  of  the  37  in  the  survey,  are 
carefully  selected  to  provide  some  measure  of  student  satisfaction  with  their  program  or 
experience  at  NPS.  The  first  question  evaluated,  Q6  states,  “Completing  a  thesis,  group 
project  or  capstone  project  was  a  valuable  component  of  my  NPS  education;”  the  second 
question  assessed,  Qll  states,  “My  academic  background  was  adequate  preparation  for 
successful  completion  of  my  program;”  the  third,  Q31  states,  “I  would  recommend  NPS 
to  other  military  officers  or  defense  civilians”  (Affairs,  2015).  These  survey  items 
provide  an  indication  of  graduates’  reflection  on  the  value  of  their  NPS  programs  and  are 
displayed  in  in  Table  11. 


Table  11.  GSS  Response  Mean  Analysis  for  Students  Graduating  AY2010-2015. 


Civilian  &  Military_ Military  Only 


Resident  DL  ALL  Resident  DL  ALL 


#  Responses 

Mean 

ft  Responses 

Mean 

#  Response 

Mean 

ft  Responses 

Mean 

ft  Responses 

Mean 

#  Responses 

Mean 

Q6 

MBA 

493 

3.54 

204 

4.43 

697 

3.84 

484 

3.52 

158 

4.51 

642 

3.77 

ALL GSBPP 

620 

3.62 

291 

4.31 

911 

3.84 

606 

3.60 

164 

4.48 

770 

3.79 

Qll 

MBA 

500 

4.28 

90 

5.00 

590 

4.38 

490 

4.28 

160 

4.57 

650 

4.35 

ALL  GSBPP 

629 

4.24 

179 

4.77 

808 

4.36 

614 

4.24 

166 

4.55 

780 

4.31 

Q31 

MBA 

488 

4.45 

184 

4.66 

672 

4.53 

478 

4.44 

146 

4.62 

624 

4.49 

ALL  GSBPP 

616 

4.47 

260 

4.63 

876 

4.52 

601 

4.46 

150 

4.63 

751 

4.50 

Source:  See  text  for  data  collection  methods  and  details.  Null  responses  are  removed  and  not  calculated  in  the 
mean  response  analysis. *2011  DL  Data  is  not  available. 


Mean  responses  for  GSBPP  and  MBA  students  in  both  the  civilian-and-military 
and  military-only  samples  followed  the  same  general  pattern.  Of  the  three  questions 
analyzed,  MBA  graduate  students  rated  as  lowest,  Q6,  which  probed  the  perceived  value 
of  the  students’  thesis,  group  project  or  capstone  project.  In  response  to  Qll,  MBA 
graduate  students  reported  feeling  that  their  academic  background  had  been  adequate  for 
successful  completion  of  their  program.  For  Q31,  on  whether  or  not  MBA  graduates 
would  recommend  NPS  to  other  military  officers  and  defense  civilians,  there  was  a  strong 
positive  response.  Of  note,  DL  students  rated  these  three  survey  items  higher  than  the 
resident  counterparts  for  both  civilian-and-military  and  military-only  samples.  Overall 
results  for  DL  MBA  students  show  that,  although  they  have  lower  response  rates,  those 
students  who  do  respond  are  more  satisfied  with  their  program  experience  than  their 

resident  counterparts.  This  finding  mirrors  that  of  all  GSBPP  DL  students. 
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It  has  been  suggested  that  people  in  an  environment  they  consider  positive  are 
motivated  and  will  perform  well  (Seppala  and  Cameron  2015).  This  view  is  only 
somewhat  supported  by  this  MBA  project’s  findings.  For  example,  the  results  of  the  DL 
MBA  GSS  survey  responses  show  both  that  DL  students  have  a  more  positive 
perspective  of  their  academic  program  experience  and  that  their  cumulative  AY2006- 
2013  grade  point  averages  are  generally  higher  than  those  of  their  resident  peers. 
However,  DL  MBA  students  have  lower  overall  graduation  rates.  That  noted,  overall 
MBA  program  performance  of  DL  MBA  students  who  do  complete  their  programs  is 
stronger  than  that  of  their  resident  student  counterparts. 

The  project’s  t-test  results  provide  some  potential  insight  into  the  between-group 
differences  of  resident  and  DL  outcomes.  For  example,  statistically  significant  between- 
group  differences  in  Resident  and  DL  samples  for  five  of  the  eight  courses  measured  by 
the  CoreQPR  appear  to  exist.  This  finding  seems  also  to  drive  the  between  group- 
differences  that  have  been  or  are  observed  in  the  CoreQPR  for  all  MBA  student  samples. 
Also,  although  no  relationship  was  found  between  entry  criteria  APC  scores  and  the  three 
grade  point  averages,  the  project’s  analysis  of  APC  scores  for  the  entire  population  of 
MBA  students  did  reveal  some  between-group  differences.  These  differences  appeared  to 
be  driven  mainly  by  the  “male”  sample  population.  Weak  correlations  were  noted  only 
between  DL  MBA  students  with  admissions  criteria  waivers  and  both  CoreQPR  grade 
point  average  and  failure  to  graduate.  The  APC  findings  support  prior  research  findings 
that  DL  students  do  not  perform  as  well  as  resident  students  when  inadequately  prepared 
(Christensen  and  Nance  2012). 

Although  all  project  findings  are  mixed,  many  of  the  results  point  to  strengths  in 
DL  student  outcomes.  For  NPS  MBA  students,  many  DL  student  success  factors  are 
equal  to,  if  not  better  than,  those  of  resident  students.  A  notable  exception  to  these 
generally  positive  findings,  however,  is  observed  for  DL  graduation  rates. 
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VI.  CONCLUSION 


My  analysis  collected  student  data  from  the  Naval  Postgraduate  School  (NPS) 
Student  Management  Database  for  resident  and  distance  learning  (DL)  students  who 
started  their  MBA  programs  in  academic  years  2006  through  2013.  The  student  data 
includes  APC  entry  requirements,  total  course,  program  and  core  course  GPAs, 
graduation  rates.  For  the  graduation  years  of  AY2010-2015  student  responses  from  the 
Graduating  Student  Survey  (GSS)  was  collected.  Following  the  collection  of  data,  I 
analyzed  the  data  using  t-tests,  correlation  and  mean  analyses,  graduation  rates,  and 
analysis  of  GSS  student  responses.  This  general  approach  to  data  analysis  is  intended  to 
provide  preliminary  observations  on  between-group  differences  for  DL  and  resident 
students  and  does  not  control  for  such  factors  as  differences  in  syllabi,  teaching  quality, 
testing  methods,  and  other  known  influences  on  student  success.  Project  analyses  were 
conducted  so  as  to  assist  in  identifying  factors  that  could  be  explored  in  further  studies 
relating  to  student  success  in  resident  and  DL  programs. 

The  literature  suggests  a  common  theme  amongst  the  majority  of  the  articles: 
hybridization  of  resident  and  DL  programs  provides  the  most  beneficial  learning 
environment  for  MBA  students.  Higher  rates  of  success  in  resident  and  hybrid  programs 
may  be  due  to  the  weighting  down  of  DL  program  success  rates  in  mathematically 
intensive  courses.  However,  the  convenience  of  DL  programs  for  busy  professionals  can 
outweigh  this  benefit  of  a  resident  program. 

This  MBA  project’s  findings  suggest  that  DL  MBA  programs  may  still  be 
working  to  find  ways  to  provide  equitable  learning  environments  and  to  address 
accreditation  challenges.  Hybridization  of  resident  and  DL  programs  may  offer  solutions 
to  the  shortcomings  of  both.  Hybrid  programs  have  the  potential  to  address  such 
accrediting  body  concerns  as  the  establishment  of  connections  with  peers  through  short 
resident  portions  of  the  program,  while  meeting  the  needs  of  busy  working  professionals. 
They  also  can  address  the  desires  of  resident  students  to  have  the  option  to  review 
lectures  and  course  materials  outside  class  meetings. 
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One  factor  that  deserves  further  exploration  for  both  resident  and  DL  MBA 
programs  at  NPS  is  the  effect  of  peers  on  individual  student  success.  Peer  effects  have 
been  studied  as  important  to  educational  outcomes,  but  no  conclusive  findings  have 
emerged  (Sacerdote  2014).  These  effects  and  how  they  may  differ  for  resident  and  DL 
students  are  particularly  important  in  the  context  of  MBA  group  projects.  Further,  they 
may  be  even  more  important  military  and  defense  MBA  programs,  and  so  deserve  further 
investigation  in  future  studies. 

The  student  data  analyzed  in  this  project  mirrors  portions  of  the  findings  from  the 
literature  review.  There  is  little  correlation  between  students  who  do  not  meet  entry 
criteria  (APC)  and  their  composite  grade  point  average;  however,  there  is  a  weak 
correlation  between  not  meeting  entry  criteria  and  being  less  likely  to  graduate.  In  the 
mean  grade  point  average  analysis,  NPS  DL  MBA  students  perform  as  well  or  better  than 
their  resident  counterparts  do.  Fodor  (2016),  in  his  analysis  of  NPS  GSBPP  students 
alone,  finds  similar  results,  with  DL  students  performing  as  well  as  resident  students. 
Nevertheless,  the  results  of  this  MBA  project’s  research  indicate  that  student  completion 
rates  for  DL  MBA  students  are  lower  by  8%  than  these  rates  for  their  resident  MBA 
peers.  These  results  are  analogous  with  both  Fodor  (2016)  and  Bacolod  and  Chaudhary 
(2016),  who  report  that  NPS  DL  students  across  all  programs  have  approximately  15% 
lower  rates  of  program  completion  than  their  resident  peers. 

The  qualitative  analysis  of  student  opinion  as  reflected  in  responses  to  the 
Graduating  Student  Survey  showed  that  student  ratings  of  their  programs  were  higher  for 
DL  than  for  resident  students.  These  results  suggest  that  MBA  DL  students  have  positive 
feelings  about  their  degree  programs  and  experiences  at  NPS,  a  measurable  outcome  of 
MBA  student  success.  While  the  results  of  all  of  these  analyses  are  mixed,  the  majority  of 
the  findings  are  positive  for  DL  student  outcomes.  Only  one  real  exception  exists,  which 
is  the  lower  rates  of  graduation  for  DL  students  seen  in  both  the  MBA  only  and  all 
GSBPP  programs. 

As  NPS  DL  MBA  courses  are  offered  synchronously,  and  students  share  a 

computer-assisted  classroom  when  in  remote  locations,  better  ways  to  foster  community 

connections  can  be  developed.  Such  innovation  may  assist  in  improving  program 
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completion  rates.  NPS  DL  MBA  student  overall  success  is  hampered  by  those  students 
who  do  not  complete  their  program,  but  a  strong  sense  of  community  might  help  those 
who  become  discouraged  in  completing  their  degrees.  Efforts  to  further  this  sense  of 
community  may  in  turn  suggest  new  ways  of  approaching  communication  and  peer 
effects  in  learning  environments.  By  lessening  the  communication  gaps  that  occur  in  the 
resident  and  DL  environments,  NPS  can  successfully  support  the  needs  of  the  Navy  and 
Armed  Forces  to  educate  our  Military  officers  while  cultivating  a  collaborative 
community  environment. 

Finally,  the  differences  observed  in  resident  and  DL  programs  may  not  so  much 
reflect  a  difference  in  student  samples,  as  the  current  stage  of  development  of  DL 
programs.  In  addition  to  the  intangible  cost  of  common  perceptions  that  DL  programs  are 
lower  in  quality,  the  traditions  and  activities  that  serve  to  connect  teaching  and  learning 
communities  in  resident  educational  environments  are  not  yet  fully  developed  in  DL 
educational  communities.  This  is  not  surprising,  as  the  evaluation  of  programs,  courses, 
faculty  and  students  in  resident  education  has  evolved  since  the  earliest  universities  in 
traditional  societies. 

In  the  1990s,  a  greater  emphasis  on  program  assessment  arose  alongside  the 
advances  in  technological  capacity  to  store  and  compare  data  (Kaplan  and  Haenlein 
2016).  Future  studies  of  resident  and  DL  delivery  modes  might  examine,  for  example, 
how  to  assess  DL  class  participation  in  new,  more  innovative  and  appropriate  ways. 
Similarly  a  more  effective  use  of  videos  and  podcasts  could  provide  new  means  for 
evaluating  DL  student  progress.  Future  studies  could  also  examine  discussion  board 
posts,  both  for  correlations  with  class  participation  and  other  course  level  scores,  and  for 
evidence  of  successful  peer  and  community  connection  strategies. 

Focus  groups  among  students  and  faculty  might  also  help  identify  attitudes  and 
approaches  able  to  improve  the  ability  for  DL  students  to  understand  material  or  grasp 
concepts  when  immediate  communications  are  not  possible.  Further  studies  could  focus 
on  better  identifying  factors  that  negatively  influence  DL  student  graduation  and  on-time 
graduation.  Future  research  could  address  if  DL  students  have  lower  program  completion 
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rates  than  resident  students  because  of  mathematic  competencies,  communication  issues, 
or  personal  issues. 

Finally,  using  findings  from  the  studies  currently  available,  efforts  should  be 
made  to  more  finely  identify  differences  in  the  way  learning  and  the  relationships  that 
support  learning  occur  in  the  DL  environment,  and  to  develop  better  assessments  for 
capturing  achievement.  Such  studies  could  help  improve  DL  educational  opportunities  in 
higher  education,  and  can  particularly  support  the  benefits  of  DL  opportunities  for 
Military  and  Defense  students. 
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APPENDIX 


Table  12.  Percentage  of  APC  Waivers  by  Start  AY  and  Student  Type 


Start  AY2006 

Start  AY2007 

Start  AY2008 

Start  AY2009 

Delivery  Mode 

Student  Type 

Student  Count 

%  of  Waivers 

Student  Count 

%  of  Waivers 

Student  Count 

%  of  Waivers 

Student  Count 

%  of  Waivers 

Res 

USN/R 

61 

10% 

50 

24% 

65 

17% 

56 

18% 

USMC 

17 

35% 

13 

31% 

13 

23% 

5 

0% 

USA/R 

9 

33% 

14 

43% 

11 

9% 

14 

50% 

USCG 

60 

28% 

9 

11% 

11 

18% 

4 

50% 

Civilian 

16 

19% 

4 

25% 

7 

43% 

9 

33% 

Res  Total 

Res 

164 

21% 

90 

27% 

107 

19% 

88 

25% 

DL 

USN/R 

48 

6% 

73 

23% 

85 

22% 

94 

18% 

USMC 

2 

0% 

3 

0% 

4 

0% 

2 

50% 

USA 

1  100% 

2 

50% 

2 

100% 

USAF 

2 

0% 

1  0% 

2 

0% 

Civilian 

2 

0% 

21 

29% 

14 

21% 

34 

50% 

DL  Total 

DL 

54 

6% 

99 

24% 

107 

21% 

132 

28% 

Grand  Total 

ALL 

218 

17% 

189 

25% 

214 

20% 

220 

27% 

Start  AY2010 

Start  AY201 1 

Start  AY2012 

Start  AY2013 

Total  AY2006- 13 

Delivery  Mode 

Student  Type 

Student  Count 

%  of  Waivers 

Student  Count 

%  of  Waivers 

Student  Count 

%  of  Waivers 

Student  Count 

%  of  Waivers 

Student  Count 

%  of  Waivers 

Res 

USN/R 

62 

11% 

53 

6% 

62 

16% 

70 

10% 

479 

14% 

USMC 

11 

18% 

13 

31% 

14 

7% 

12 

8% 

98 

21% 

USA/R 

15 

27% 

11 

45% 

32 

19% 

18 

28% 

124 

30% 

USCG 

26 

19% 

11 

55% 

1 

0% 

20 

20% 

142 

26% 

Civilian 

8 

38% 

8 

13% 

6 

17% 

15 

7% 

73 

22% 

Res  Total 

Res 

122 

17% 

96 

20% 

116 

16% 

138 

13% 

921 

19% 

DL 

USN/R 

76 

33% 

103 

22% 

88 

27% 

68 

22% 

635 

23% 

USMC 

4 

25% 

5 

0% 

3 

33% 

23 

13% 

USA 

1  0% 

4 

0% 

1 

0% 

ii 

36% 

USAF 

3 

67% 

1 

0% 

9 

22% 

Civilian 

49 

53% 

24 

50% 

35 

23% 

34 

38% 

213 

40% 

DL  Total 

DL 

128 

41% 

132 

27% 

133 

24% 

106 

27% 

891 

27% 

Grand  Total 

ALL 

250 

30% 

228 

24% 

249 

20% 

244 

19% 

1812 

23% 

Source:  See  text  for  data  collection  methods  and  details. 


Figure  5.  MBA  Graduation  Type  by  All  Student  Population 

Sample  and  Res/DL. 


MBA  Graduation  Rates  Start  AY2006-AY2013 


Source:  See  text  for  data  collection  methods  and  details.  Graduate  data  collected  displays  the 
proportions  of  the  three  graduate  categories.  On-Time  is  defined  as  students  who  graduate  at  the 
time  expected  without  an  extension.  Late  is  defined  as  students  who  graduate  beyond  the 
expected  graduation  date,  with  extensions.  Non-Grad  is  defined  as  a  student  who  has  yet  to 
graduate. 
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Figure  6.  MBA  Graduation  Type  by  Military  Only  Student  Sample  Population 

and  Res/DL. 


Military  MBA  Graduation  Rates  Start  AY2006-AY2013 


Late 


Source:  See  text  for  data  collection  methods  and  details.  Graduate  data  collected  displays  the 
proportions  of  the  three  graduate  categories.  On-Time  is  defined  as  students  who  graduate  at  the 
time  expected  without  an  extension.  Late  is  defined  as  students  who  graduate  beyond  the 
expected  graduation  date,  with  extensions.  Non-Grad  is  defined  as  a  student  who  has  yet  to 
graduate. 
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